-- Logs begin at Mon 2022-09-26 17:00:36 UTC, end at Mon 2022-09-26 18:28:59 UTC. -- Sep 26 17:42:59.555239 dv-cinder-859319-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 26 17:42:59.565789 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 26 17:42:59.568005 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Sep 26 17:42:59 2022] *** Sep 26 17:42:59.568005 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 26 17:42:59.568005 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 26 17:42:59.568005 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: nodename: dv-cinder-859319-1-iscsi Sep 26 17:42:59.568005 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: machine: x86_64 Sep 26 17:42:59.568005 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: clock source: unix Sep 26 17:42:59.568005 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: pcre jit disabled Sep 26 17:42:59.568005 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: detected number of CPU cores: 4 Sep 26 17:42:59.568005 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: current working directory: / Sep 26 17:42:59.568005 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: detected binary path: /usr/bin/uwsgi-core Sep 26 17:42:59.568528 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: your processes number limit is 47680 Sep 26 17:42:59.568528 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: your memory page size is 4096 bytes Sep 26 17:42:59.568528 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: detected max file descriptor number: 2048 Sep 26 17:42:59.568528 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: lock engine: pthread robust mutexes Sep 26 17:42:59.568528 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: thunder lock: enabled Sep 26 17:42:59.568528 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 26 17:42:59.568528 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 26 17:42:59.608460 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: Python main interpreter initialized at 0x55bc260769c0 Sep 26 17:42:59.608460 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: python threads support enabled Sep 26 17:42:59.608460 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: your server socket listen backlog is limited to 100 connections Sep 26 17:42:59.608460 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: your mercy for graceful operations on workers is 90 seconds Sep 26 17:42:59.608791 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: mapped 403077 bytes (393 KB) for 2 cores Sep 26 17:42:59.608864 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: *** Operational MODE: preforking *** Sep 26 17:42:59.609139 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: *** uWSGI is running in multiple interpreter mode *** Sep 26 17:42:59.609139 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: spawned uWSGI master process (pid: 81488) Sep 26 17:42:59.609334 dv-cinder-859319-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 26 17:42:59.610430 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: spawned uWSGI worker 1 (pid: 81499, cores: 1) Sep 26 17:42:59.610430 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: spawned uWSGI worker 2 (pid: 81500, cores: 1) Sep 26 17:42:59.610430 dv-cinder-859319-1-iscsi devstack@keystone.service[81488]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 26 17:43:03.939771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 17:43:03.939771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: from cryptography.utils import int_from_bytes Sep 26 17:43:03.940579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 17:43:03.940579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: from cryptography.utils import int_from_bytes Sep 26 17:43:03.945826 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 17:43:03.945826 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: from cryptography.utils import int_from_bytes Sep 26 17:43:03.946311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 26 17:43:03.946311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: from cryptography.utils import int_from_bytes Sep 26 17:43:03.991904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 26 17:43:03.992107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 26 17:43:03.992302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] command line args: [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 26 17:43:03.992577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 26 17:43:03.992758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ================================================================================ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 26 17:43:03.993018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] admin_token = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.993286 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] config_dir = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.993459 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.993644 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] config_source = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.993824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] debug = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.994022 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.995173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.995173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.995173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.995173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.995432 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.995495 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_config_append = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.995700 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.995863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 26 17:43:03.995969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_dir = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.996049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.996118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 26 17:43:03.996256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] command line args: [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 26 17:43:03.996358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.996505 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.996594 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 26 17:43:03.996703 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ================================================================================ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 26 17:43:03.996780 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.996967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] admin_token = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.997050 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.997135 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.997213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] config_dir = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.997316 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.997400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.997563 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.997648 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] config_source = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.997759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] debug = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.997875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.997958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.998057 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.998194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.998295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.998376 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.998470 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.998622 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.998700 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.998878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.998957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.999127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.999237 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.999418 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_config_append = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.999511 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.999595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.999814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_dir = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:03.999972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.67/identity {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.000061 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.000144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] publish_errors = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.000216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.000380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.000460 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.000607 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.000709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.000793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.001072 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.001180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.001254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.001323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.001398 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.001478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] standard_threads = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.001590 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.001705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.001843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.001927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.002074 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.002157 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.002252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] use_journal = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.002329 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.002445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] use_json = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.002531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.002656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] use_stderr = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.002761 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.002855 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] use_syslog = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.003015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.003091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.003264 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.003348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.003523 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.003599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.003810 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.004053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.004140 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.67/identity {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.004241 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] publish_errors = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.004317 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.004469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.004556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.004705 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.004784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.004891 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.004989 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.005144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.005229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.005385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.005461 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.005628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] standard_threads = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.005710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.005866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.005950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.006252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.006926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.007014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.007092 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.007223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] use_journal = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.007408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] use_json = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.007526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.007612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] use_stderr = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.010667 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.010749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] use_syslog = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.010819 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.010898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 26 17:43:04.010976 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.011053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.011124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.011193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.011273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.011344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.011418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.011489 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.011564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.011635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.011745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.011818 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.011887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.011998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.012079 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.012152 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.012229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.012305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.012376 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.012446 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.012521 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.012595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.012715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.012789 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.012858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.012931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.013002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.013076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.013156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.013156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.013283 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.013351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.013420 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.013494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.013571 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.013641 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.013715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.013791 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.013866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.013935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.014011 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.014095 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.014095 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.014221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.014290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.014618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.014701 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.014772 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.014847 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.014917 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.014985 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.015054 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.015133 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.015204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.015272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.015340 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.015414 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.015483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.015559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.015629 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.015732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.015814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.015885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.015979 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.016065 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.016219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.016296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.016459 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.016536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.016704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.016857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.016931 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.017006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.017166 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.017244 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.017404 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.017483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.017642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.017721 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.017875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.017958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.018125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.018205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.018361 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.018438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.018735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.018861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.018968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.019065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.019173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.019270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.019370 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.019466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.019560 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.019715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.019792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.019910 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.019996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.020069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.020243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.020333 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.020484 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.020569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.020723 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.020844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.020967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.021051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.021206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.021289 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.021448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.021534 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.021677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.021828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.021940 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.022285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.022374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.022537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.022624 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.022777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.022855 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.023027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.023114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.023266 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.023353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.023509 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.023597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.023785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.023862 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.023993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.024076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.024218 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.024308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.024455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.024534 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.024692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.024777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.024932 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.025022 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.025208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.025288 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.025451 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.025535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.025695 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.025781 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.025935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.026023 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.026173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.026251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.026407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.026484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.026646 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.026724 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.026881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.026958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.027112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.027191 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.027343 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.027422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.027573 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.027671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.027833 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.027913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.028065 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.028145 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.028306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.028385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.028541 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.028621 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.029033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.029172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.029289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.029365 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.029431 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.029500 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.029579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.029652 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.029743 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.029855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.029972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.external = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.030111 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.030198 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.030359 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.030483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.030601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.030715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.030854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.030940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.password = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.031096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.031175 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.token = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.031338 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.031422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.031578 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.031718 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.031840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.031920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.032076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.032185 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.032314 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.032395 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.032553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.032640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.032782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.032867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.033019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.033104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.033258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.033345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.033504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.033583 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.033742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.033819 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.033986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.034072 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.034232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.034312 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.034467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.034552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.034707 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.external = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.034793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.034942 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.035051 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.035180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.035259 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.035435 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.035523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.035712 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.password = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.036068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.036184 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.token = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.036271 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.036345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.036423 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.036502 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.036581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.036698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.036778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.036933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.037024 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.037169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.037255 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.037471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.037546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.037719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.037882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.037967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.038119 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.038290 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.038375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.038548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.038691 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.038831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.038920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.039054 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.039132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.039303 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.039383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.039536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.039615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.039808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.039888 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.040045 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.040126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.040294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.040373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.040543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.040625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.040779 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.041002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.041140 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.041218 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.041290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.041423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.041515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.041673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.041759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.041931 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.042009 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.042170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.042256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.042402 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.042480 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.042633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.042713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.042865 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.042953 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.043109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.043189 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.043354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.043433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.043796 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.043892 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.043968 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.044040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.044111 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.044182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.044341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.044430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.044583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.044662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.044822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.044901 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.045062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.045186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.045322 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.045407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.045559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.045637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.045748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.045881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.045988 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.046123 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.046226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.046378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.046476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.046616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.046697 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.046859 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.046938 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.047113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.047201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.047370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.047449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.047608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.047711 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.047871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.047964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.048109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.048194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.048347 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.048426 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.048577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.048732 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.048808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.048880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.049039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.049119 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.049281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.049368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.049518 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.049599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.049756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.049835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.050005 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.050094 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.050259 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.050366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.050504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.050602 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.050747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.050834 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.050989 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.051076 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.051235 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.051314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.051479 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.051566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.051756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.051841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.052004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.052187 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.052286 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.052416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.052518 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.052598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.052710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.052791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.053455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.053616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.053725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.053800 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.054015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.054105 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.054178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.054336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.054418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.054573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.054652 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.054820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.054899 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.055064 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.055143 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.055300 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.055409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.055532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.055611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.055798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.055891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.056039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.056142 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.056284 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.056364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.056519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.056598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.056756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.056834 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.056983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.057060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.057229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.057309 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.057465 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.057550 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.057700 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.057779 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.057935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.058016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.058168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.058247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.058413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.058490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.058657 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.058803 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.058884 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.058956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.059207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.059289 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.059358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.059491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.059571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.059780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.059863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.060024 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.060117 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.060261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.060341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.060504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.060584 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.060743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.060830 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.060971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.061050 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.061205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.061284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.061443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.061524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.061694 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.061772 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.061939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.062023 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.062172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.062253 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.062419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.062499 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.062653 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.062731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.062889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.062969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.063131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.063363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.063442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.063601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.063780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.063945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] role.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.064035 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.064120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] role.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.064200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.064348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.064506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.064585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.064752 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.064831 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.065016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.065103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.065255 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.065334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.065507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.065586 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.065753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.065840 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.065996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.066074 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.066236 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.066332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.066454 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.066567 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.066701 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.066780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.066947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.067032 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.067191 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.067278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.067434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.067513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.067706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.067864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.067964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.068103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.068211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.068335 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.068585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.069188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.069404 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.069517 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.069656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.069846 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.069935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.070104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.070183 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.070350 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.070430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.070597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.070675 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.070829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.070908 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] role.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.071060 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.071140 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] role.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.071290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.071369 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.071540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.071628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.071820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.071899 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.072075 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.072154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.072312 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.072404 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.072546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.072626 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.072775 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.072854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.073009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.073161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.073236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.073308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.073478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.073557 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.073712 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.073800 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.073968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.074048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.074215 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.074294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.074451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.074611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.074713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.074917 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.074999 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.075159 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.075238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.075393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.075474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.075629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.075778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.075883 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.075963 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.076127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.076208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.076388 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.076466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.076613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.076700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.076855 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.076944 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.077119 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.077199 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.077351 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.077428 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.077590 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.077676 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.077828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.077907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.078061 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.078139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.078299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.078386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.078532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.078611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.078766 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.078845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.079013 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.079105 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.079232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.079368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.079473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.079614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.079749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.079884 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.079965 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.080141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.080220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.080386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.080466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.080641 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.080721 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.080885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.080969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.081115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.081194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.081355 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.081436 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.081593 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.081673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.081764 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 26 17:43:04.081901 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.082145 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.082391 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.082630 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.082880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.083165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.083400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.083670 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.083943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.084245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.084706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.085053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.085340 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.085596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.085844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.086091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.086335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.086581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.086835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.087082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.087334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.087577 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.087870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.088114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.088365 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.088711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.088963 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.089206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 26 17:43:04.089382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 26 17:43:04.110894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.111118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.111337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.115129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.115361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.116153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.120177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.124607 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.148689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.148906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.156010 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.156249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.215039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.215212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.264004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.264233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.266597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.266872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.425919 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.426107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.426181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.426333 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.426428 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.426556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.426665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.426775 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.426858 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.426991 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.427073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.427209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.455724 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.455863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.455950 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.456016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.456179 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.456282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.456401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.456484 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.456617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.456710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.456833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.456916 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.457047 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.457129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.469508 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.469640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.482429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.482550 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.482680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.482923 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.487377 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.487468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.487544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.487616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.487707 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.487795 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.487870 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.487946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.488018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.488079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 26 17:43:04.503328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.503554 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 17:43:04.503639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.503840 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 17:43:04.504060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.504182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.504278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 17:43:04.504381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 26 17:43:04.504527 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.504657 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.504792 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.504927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.505060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.505188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.505281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.505464 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.515895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.516177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.516263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.516449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.516533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.516735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.516819 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.517033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.524283 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.524785 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.527154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.527438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.527743 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.527827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.528027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.528109 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.528312 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.528407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.528574 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.528654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.528844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.528923 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.529118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.529206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.529391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.529470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.529666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.529745 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.529935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.530014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.530212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.530483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.552911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.553456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.554488 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.554773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.554908 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.555191 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.559436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.559755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.559899 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.560041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.560185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.560466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.566343 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.566630 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.566843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.567129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.573245 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.573479 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.573567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.573776 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.573858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.574065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.574147 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.574344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.574428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.574619 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.574702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.574891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.574981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.575161 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.575244 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.575429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.591233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.591519 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.591607 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.591818 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.595367 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.595625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.598590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.598844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.598927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.599139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.603335 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.603438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.606492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.606578 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.609730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.609822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.610031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.610127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.610317 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.610399 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.610596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.610676 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.610872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.610954 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.611147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.611229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.627422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.627602 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.627735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.627915 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.627997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.628185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.628284 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.628451 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.635729 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.636122 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.637258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.637651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.638779 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.639064 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.639170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.639722 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.642032 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.642319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.642402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.642607 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.642729 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.642887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.642968 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.643166 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.643281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.643560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.654511 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.654798 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.655082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.655165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.655365 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.655448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.655707 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.655802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.655990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.656073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.656319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.656400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.656641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.656937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.671690 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.672596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.674555 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.675501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.677536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.678933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.679617 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.680401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.681128 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.681410 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.681799 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.681881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.682118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.682523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.688219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.688904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.689671 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.690357 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.692578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.692979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.693216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.693377 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.693531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.693737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.693982 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.694256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.701503 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.701796 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.702006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.702090 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.702296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.702573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.708895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.709726 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.710098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.710376 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 26 17:43:04.710471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.710646 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.710729 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.710929 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.711010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.711203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.711284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.711477 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.711564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.711773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.711859 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.712045 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.712126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.712322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.712416 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.712595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.712676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.712864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.712944 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.713133 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.713258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.713402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.713669 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.713980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 26 17:43:04.747330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55bc260769c0 pid: 81500 (default app) Sep 26 17:43:04.748750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55bc260769c0 pid: 81499 (default app) Sep 26 17:43:04.750557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f812c-6fd4-4a7b-8b03-a565f50782fa None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:04.750843 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f812c-6fd4-4a7b-8b03-a565f50782fa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:04.751112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4f812c-6fd4-4a7b-8b03-a565f50782fa None None] PATH_INFO: `/v3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:04.755256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1/1] 10.222.0.67 () {50 vars in 791 bytes} [Mon Sep 26 17:43:04 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:12.396805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8ed07-1b36-440e-b205-0dadc10e32ec None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:12.397303 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8ed07-1b36-440e-b205-0dadc10e32ec None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:12.397442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f8ed07-1b36-440e-b205-0dadc10e32ec None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:12.401765 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1/2] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:43:12.407152 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389359c8-eaf1-42b4-a680-d58cbd11272e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:12.407589 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389359c8-eaf1-42b4-a680-d58cbd11272e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:12.407716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389359c8-eaf1-42b4-a680-d58cbd11272e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:12.478481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: INFO dbcounter [None req-389359c8-eaf1-42b4-a680-d58cbd11272e None None] Registered counter for database keystone Sep 26 17:43:12.497697 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG oslo_db.sqlalchemy.engines [None req-389359c8-eaf1-42b4-a680-d58cbd11272e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81500) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 26 17:43:12.503823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writer thread running {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 26 17:43:12.681759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG passlib.handlers.bcrypt [None req-389359c8-eaf1-42b4-a680-d58cbd11272e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81500) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 26 17:43:12.682012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG passlib.handlers.bcrypt [None req-389359c8-eaf1-42b4-a680-d58cbd11272e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81500) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 26 17:43:12.739718 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-389359c8-eaf1-42b4-a680-d58cbd11272e None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:12.741463 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-389359c8-eaf1-42b4-a680-d58cbd11272e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:12.867445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 2/3] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:12 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 26 17:43:13.326985 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8ed07-1b36-440e-b205-0dadc10e32ec None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:13.397898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: INFO dbcounter [None req-10f8ed07-1b36-440e-b205-0dadc10e32ec None None] Registered counter for database keystone Sep 26 17:43:13.419098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG oslo_db.sqlalchemy.engines [None req-10f8ed07-1b36-440e-b205-0dadc10e32ec None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81499) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 26 17:43:13.425279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writer thread running {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 26 17:43:13.543871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f8ed07-1b36-440e-b205-0dadc10e32ec None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:13.548076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194c9e8-6146-4935-9601-c7e68fb780c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:13.548700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194c9e8-6146-4935-9601-c7e68fb780c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:13.548982 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194c9e8-6146-4935-9601-c7e68fb780c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:13.549255 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194c9e8-6146-4935-9601-c7e68fb780c7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:13.557563 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194c9e8-6146-4935-9601-c7e68fb780c7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:13.608674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8194c9e8-6146-4935-9601-c7e68fb780c7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 26 17:43:13.608961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194c9e8-6146-4935-9601-c7e68fb780c7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:13.618675 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 2/4] 10.222.0.67 () {58 vars in 1324 bytes} [Mon Sep 26 17:43:13 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 17:43:13.624622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389359c8-eaf1-42b4-a680-d58cbd11272e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:13.642372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389359c8-eaf1-42b4-a680-d58cbd11272e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:13.647445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702cb94-7019-413c-9216-0499135925a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:13.648588 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702cb94-7019-413c-9216-0499135925a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:13.648872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702cb94-7019-413c-9216-0499135925a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:13.649144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702cb94-7019-413c-9216-0499135925a8 admin admin] PATH_INFO: `/v3/projects/1537d57f83734659b1130d2661d44ac2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:13.650736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702cb94-7019-413c-9216-0499135925a8 admin admin] RBAC: Authorizing `identity:get_project(project_id=1537d57f83734659b1130d2661d44ac2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:13.704369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b702cb94-7019-413c-9216-0499135925a8 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 26 17:43:13.704717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702cb94-7019-413c-9216-0499135925a8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:13.710166 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 3/5] 10.222.0.67 () {56 vars in 1207 bytes} [Mon Sep 26 17:43:13 2022] GET /identity/v3/projects/1537d57f83734659b1130d2661d44ac2 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:15.844139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f0e88-0864-44b2-8991-58db3c86b01e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:15.844656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f0e88-0864-44b2-8991-58db3c86b01e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:15.844656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056f0e88-0864-44b2-8991-58db3c86b01e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:15.848759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 3/6] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:15.854177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853aa3ff-ae63-4c65-95f3-757c67ab9e15 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:15.854571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853aa3ff-ae63-4c65-95f3-757c67ab9e15 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:15.854716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853aa3ff-ae63-4c65-95f3-757c67ab9e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:15.874764 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-853aa3ff-ae63-4c65-95f3-757c67ab9e15 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:15.876302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-853aa3ff-ae63-4c65-95f3-757c67ab9e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:15.884629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 4/7] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:15 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 26 17:43:16.152360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f0e88-0864-44b2-8991-58db3c86b01e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:16.164846 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056f0e88-0864-44b2-8991-58db3c86b01e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:16.168986 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ec07e-df5d-4fe0-a356-1633165be4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:16.169581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ec07e-df5d-4fe0-a356-1633165be4f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:16.169860 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ec07e-df5d-4fe0-a356-1633165be4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:16.170129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ec07e-df5d-4fe0-a356-1633165be4f3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:16.177618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ec07e-df5d-4fe0-a356-1633165be4f3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:16.179477 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ec07e-df5d-4fe0-a356-1633165be4f3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:16.188059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 4/8] 10.222.0.67 () {58 vars in 1324 bytes} [Mon Sep 26 17:43:16 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 17:43:16.193591 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853aa3ff-ae63-4c65-95f3-757c67ab9e15 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:16.202652 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853aa3ff-ae63-4c65-95f3-757c67ab9e15 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:16.206935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31376c2b-9341-49b2-bbb6-ca5321a1f6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:16.207557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31376c2b-9341-49b2-bbb6-ca5321a1f6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:16.207871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31376c2b-9341-49b2-bbb6-ca5321a1f6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:16.208147 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31376c2b-9341-49b2-bbb6-ca5321a1f6c1 admin admin] PATH_INFO: `/v3/users/e71ead94768141dc94d90c67625c3ac2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:16.210197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31376c2b-9341-49b2-bbb6-ca5321a1f6c1 admin admin] RBAC: Authorizing `identity:get_user(user_id=e71ead94768141dc94d90c67625c3ac2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:16.212062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31376c2b-9341-49b2-bbb6-ca5321a1f6c1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:16.213575 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 5/9] 10.222.0.67 () {56 vars in 1198 bytes} [Mon Sep 26 17:43:16 2022] GET /identity/v3/users/e71ead94768141dc94d90c67625c3ac2 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:18.463781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6e104-2812-472e-be15-bb01480fa3a6 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:18.464358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6e104-2812-472e-be15-bb01480fa3a6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:18.464358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e6e104-2812-472e-be15-bb01480fa3a6 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:18.465667 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 5/10] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:18.470862 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b0876-ff8c-4f33-afa7-94564417433c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:18.471284 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b0876-ff8c-4f33-afa7-94564417433c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:18.471488 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b0876-ff8c-4f33-afa7-94564417433c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:18.490613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-873b0876-ff8c-4f33-afa7-94564417433c None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:18.491928 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-873b0876-ff8c-4f33-afa7-94564417433c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:18.516520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c3024-e0f1-40ff-8ffc-d4f518ba0ac8 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:18.516709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c3024-e0f1-40ff-8ffc-d4f518ba0ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:18.516984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c3024-e0f1-40ff-8ffc-d4f518ba0ac8 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:18.518302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 6/11] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:43:18.523344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd58ae4-cb92-4bea-af69-3a64655785fa None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:18.523629 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd58ae4-cb92-4bea-af69-3a64655785fa None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:18.523933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd58ae4-cb92-4bea-af69-3a64655785fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:18.566415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG passlib.handlers.bcrypt [None req-3cd58ae4-cb92-4bea-af69-3a64655785fa None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81499) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 26 17:43:18.566822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG passlib.handlers.bcrypt [None req-3cd58ae4-cb92-4bea-af69-3a64655785fa None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81499) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 26 17:43:18.576241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 6/12] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:18.616859 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3cd58ae4-cb92-4bea-af69-3a64655785fa None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:18.618473 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3cd58ae4-cb92-4bea-af69-3a64655785fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:18.717203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 7/13] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:43:18.839817 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b0876-ff8c-4f33-afa7-94564417433c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:18.864018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:18.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b0876-ff8c-4f33-afa7-94564417433c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:18.983037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f45cdd-d3f9-48b8-b880-bf5779e3e802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:18.983675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f45cdd-d3f9-48b8-b880-bf5779e3e802 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:18.983976 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f45cdd-d3f9-48b8-b880-bf5779e3e802 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:18.984251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f45cdd-d3f9-48b8-b880-bf5779e3e802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:18.997449 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd58ae4-cb92-4bea-af69-3a64655785fa None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:19.069278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f45cdd-d3f9-48b8-b880-bf5779e3e802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:19.070765 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f45cdd-d3f9-48b8-b880-bf5779e3e802 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:19.094407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:19.103158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd58ae4-cb92-4bea-af69-3a64655785fa None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:19.105605 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:19.140951 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3aa6e-365c-4588-bc28-c63fe2d119ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:19.142066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3aa6e-365c-4588-bc28-c63fe2d119ad None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:19.142351 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3aa6e-365c-4588-bc28-c63fe2d119ad None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:19.142625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa3aa6e-365c-4588-bc28-c63fe2d119ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:19.155372 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa3aa6e-365c-4588-bc28-c63fe2d119ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-afa3aa6e-365c-4588-bc28-c63fe2d119ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:19.164668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:19.165706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:19.166595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:19.166595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:19.166595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:19.166595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:19.166595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:19.166595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:19.166595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:19.166595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:19.166595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:19.166595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:19.166595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:19.166595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:43:19.166595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 8/14] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:43:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:43:19.171892 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3aa6e-365c-4588-bc28-c63fe2d119ad None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:19.174516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 7/15] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:43:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:43:19.180704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f45cdd-d3f9-48b8-b880-bf5779e3e802 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:19.272958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa3aa6e-365c-4588-bc28-c63fe2d119ad None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:19.285089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f45cdd-d3f9-48b8-b880-bf5779e3e802 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:19.308087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:19.309509 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8464c8-f3c8-4b9a-972a-3fab86a9cb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:19.310405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8464c8-f3c8-4b9a-972a-3fab86a9cb6d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:19.310685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8464c8-f3c8-4b9a-972a-3fab86a9cb6d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:19.310961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8464c8-f3c8-4b9a-972a-3fab86a9cb6d None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:19.311627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8464c8-f3c8-4b9a-972a-3fab86a9cb6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:19.311958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8464c8-f3c8-4b9a-972a-3fab86a9cb6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:19.313392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8464c8-f3c8-4b9a-972a-3fab86a9cb6d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:19.319675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1889399-04ba-41fc-a6de-dc12a191f5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:19.320398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1889399-04ba-41fc-a6de-dc12a191f5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:19.320642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1889399-04ba-41fc-a6de-dc12a191f5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:19.320913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1889399-04ba-41fc-a6de-dc12a191f5b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:19.325456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 9/16] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:19.331247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1889399-04ba-41fc-a6de-dc12a191f5b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:19.331387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8464c8-f3c8-4b9a-972a-3fab86a9cb6d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:19.332798 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1889399-04ba-41fc-a6de-dc12a191f5b9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c1889399-04ba-41fc-a6de-dc12a191f5b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:43:19.351747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:19.352785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:43:19.353581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:43:19.353581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:43:19.353581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:43:19.353581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 8/17] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:43:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:43:19.358498 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1889399-04ba-41fc-a6de-dc12a191f5b9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:19.358708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:19.435069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8464c8-f3c8-4b9a-972a-3fab86a9cb6d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:19.462371 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1889399-04ba-41fc-a6de-dc12a191f5b9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:19.473475 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2272150-a748-46cd-9e09-f32c53b8e8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:19.474110 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2272150-a748-46cd-9e09-f32c53b8e8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:19.474571 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2272150-a748-46cd-9e09-f32c53b8e8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:19.474820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2272150-a748-46cd-9e09-f32c53b8e8e7 None admin] PATH_INFO: `/v3/users/e71ead94768141dc94d90c67625c3ac2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:19.476335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2272150-a748-46cd-9e09-f32c53b8e8e7 None admin] RBAC: Authorizing `identity:get_user(user_id=e71ead94768141dc94d90c67625c3ac2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:19.477797 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2272150-a748-46cd-9e09-f32c53b8e8e7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:19.479366 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 10/18] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:19 2022] GET /identity/v3/users/e71ead94768141dc94d90c67625c3ac2 => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:19.484693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2272150-a748-46cd-9e09-f32c53b8e8e7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:19.505377 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca521d30-142c-41ae-9714-c7aaafea64f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:19.506565 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca521d30-142c-41ae-9714-c7aaafea64f5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:19.506864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca521d30-142c-41ae-9714-c7aaafea64f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:19.507197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca521d30-142c-41ae-9714-c7aaafea64f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:19.507881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca521d30-142c-41ae-9714-c7aaafea64f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:19.508188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca521d30-142c-41ae-9714-c7aaafea64f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:19.509585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca521d30-142c-41ae-9714-c7aaafea64f5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:19.527496 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 9/19] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:19.584912 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:19.592860 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2272150-a748-46cd-9e09-f32c53b8e8e7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:19.631940 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ecb7ba-d16a-49be-bb0f-55f42662b63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:19.632606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecb7ba-d16a-49be-bb0f-55f42662b63a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:19.632894 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecb7ba-d16a-49be-bb0f-55f42662b63a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:19.633164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ecb7ba-d16a-49be-bb0f-55f42662b63a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:19.634342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecb7ba-d16a-49be-bb0f-55f42662b63a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:19.636028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ecb7ba-d16a-49be-bb0f-55f42662b63a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:19.637796 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 11/20] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:43:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:19.644052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca521d30-142c-41ae-9714-c7aaafea64f5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:19.682760 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:19.752952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca521d30-142c-41ae-9714-c7aaafea64f5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:19.790620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1cb2c-dbce-47d8-8d08-98d62b2640cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:19.791471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1cb2c-dbce-47d8-8d08-98d62b2640cd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:19.791941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1cb2c-dbce-47d8-8d08-98d62b2640cd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:19.792220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1cb2c-dbce-47d8-8d08-98d62b2640cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:19.792962 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1cb2c-dbce-47d8-8d08-98d62b2640cd None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.domain.id=default, user.id=e71ead94768141dc94d90c67625c3ac2) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:19.793269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1cb2c-dbce-47d8-8d08-98d62b2640cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:19.794734 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1cb2c-dbce-47d8-8d08-98d62b2640cd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:19.802038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 10/21] 10.222.0.67 () {56 vars in 1322 bytes} [Mon Sep 26 17:43:19 2022] GET /identity/v3/role_assignments?user.id=e71ead94768141dc94d90c67625c3ac2&scope.domain.id=default&role.id=07626899472b4e32adc0c71ce796a25d => generated 233 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:22.006977 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ff4e8-ac73-41a0-8990-a02987909a3c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:22.007494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ff4e8-ac73-41a0-8990-a02987909a3c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:22.007494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107ff4e8-ac73-41a0-8990-a02987909a3c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:22.008878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 12/22] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:22.014709 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078abd9-a9cf-4a59-a18a-44e75cca65a5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:22.015523 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078abd9-a9cf-4a59-a18a-44e75cca65a5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:22.016033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078abd9-a9cf-4a59-a18a-44e75cca65a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:22.038464 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c078abd9-a9cf-4a59-a18a-44e75cca65a5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:22.040043 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c078abd9-a9cf-4a59-a18a-44e75cca65a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:22.138138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 11/23] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:22.730750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ff4e8-ac73-41a0-8990-a02987909a3c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:22.776995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1f77c-a55f-495a-bb08-41a89be7d82f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:22.777324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1f77c-a55f-495a-bb08-41a89be7d82f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:22.777555 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb1f77c-a55f-495a-bb08-41a89be7d82f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:22.779195 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 12/24] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:22.785860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cc544-f373-4122-869d-f0a2e77b3dd0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:22.786158 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cc544-f373-4122-869d-f0a2e77b3dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:22.786407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867cc544-f373-4122-869d-f0a2e77b3dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:22.814704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-867cc544-f373-4122-869d-f0a2e77b3dd0 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:22.816481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-867cc544-f373-4122-869d-f0a2e77b3dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:22.858566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:22.930948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107ff4e8-ac73-41a0-8990-a02987909a3c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:22.964996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 13/25] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:22.967946 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4becedd-caa2-499d-a4ff-1bc965ff4591 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:22.968442 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4becedd-caa2-499d-a4ff-1bc965ff4591 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:22.968928 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4becedd-caa2-499d-a4ff-1bc965ff4591 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:22.970540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 14/26] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:22.977048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b32df6-077e-4732-a627-31be5f0a7f17 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:22.977638 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b32df6-077e-4732-a627-31be5f0a7f17 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:22.978138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b32df6-077e-4732-a627-31be5f0a7f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:23.001369 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e2f13-7e0f-4aee-95a2-f882502aae78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:23.002128 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e2f13-7e0f-4aee-95a2-f882502aae78 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:23.002415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e2f13-7e0f-4aee-95a2-f882502aae78 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:23.002690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e2f13-7e0f-4aee-95a2-f882502aae78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:23.017821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e1b32df6-077e-4732-a627-31be5f0a7f17 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:23.019042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e1b32df6-077e-4732-a627-31be5f0a7f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:23.038869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e2f13-7e0f-4aee-95a2-f882502aae78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:23.040403 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e2f13-7e0f-4aee-95a2-f882502aae78 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a03e2f13-7e0f-4aee-95a2-f882502aae78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:43:23.077409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:23.078288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:43:23.079077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:43:23.079077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:43:23.079077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:43:23.082867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 13/27] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:43:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:23.108946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e2f13-7e0f-4aee-95a2-f882502aae78 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:23.128090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:23.149576 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 15/28] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:23.251496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e2f13-7e0f-4aee-95a2-f882502aae78 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:23.277485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658526a-9458-4a6e-9035-9589d4af4774 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:23.278019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658526a-9458-4a6e-9035-9589d4af4774 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:23.278464 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658526a-9458-4a6e-9035-9589d4af4774 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:23.279958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 16/29] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:43:23.286107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f33378-5e25-4894-bd71-b8eb86c83de6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:23.286383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f33378-5e25-4894-bd71-b8eb86c83de6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:23.287213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f33378-5e25-4894-bd71-b8eb86c83de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:23.322204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-20f33378-5e25-4894-bd71-b8eb86c83de6 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:23.323413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-20f33378-5e25-4894-bd71-b8eb86c83de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:23.325951 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805f32f-ced1-4ec6-9a5e-1de0c05a1ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:23.326569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805f32f-ced1-4ec6-9a5e-1de0c05a1ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:23.326846 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805f32f-ced1-4ec6-9a5e-1de0c05a1ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:23.327119 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805f32f-ced1-4ec6-9a5e-1de0c05a1ff4 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:23.327788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805f32f-ced1-4ec6-9a5e-1de0c05a1ff4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:23.328090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805f32f-ced1-4ec6-9a5e-1de0c05a1ff4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:23.329474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805f32f-ced1-4ec6-9a5e-1de0c05a1ff4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:23.369985 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 14/30] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:43:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:23.375935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805f32f-ced1-4ec6-9a5e-1de0c05a1ff4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:23.494426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 17/31] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:23.495291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f33378-5e25-4894-bd71-b8eb86c83de6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:23.542637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:23.549257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:23.569419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805f32f-ced1-4ec6-9a5e-1de0c05a1ff4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:23.608441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac374f5-b89b-4acd-9812-bf5860ac53ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:23.609227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac374f5-b89b-4acd-9812-bf5860ac53ef None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:23.609472 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac374f5-b89b-4acd-9812-bf5860ac53ef None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:23.609746 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac374f5-b89b-4acd-9812-bf5860ac53ef None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:23.610393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac374f5-b89b-4acd-9812-bf5860ac53ef None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:23.611992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac374f5-b89b-4acd-9812-bf5860ac53ef None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:23.645469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f33378-5e25-4894-bd71-b8eb86c83de6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:23.683912 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aeafde-246c-4b14-8ee7-2ec50e07134f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:23.684583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aeafde-246c-4b14-8ee7-2ec50e07134f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:23.684864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aeafde-246c-4b14-8ee7-2ec50e07134f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:23.685135 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aeafde-246c-4b14-8ee7-2ec50e07134f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:23.712007 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aeafde-246c-4b14-8ee7-2ec50e07134f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-69aeafde-246c-4b14-8ee7-2ec50e07134f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:23.720476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:23.721422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:23.722154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:23.722154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:23.722154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:23.722154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:23.722154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:23.722154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:23.722154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:23.722154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:23.722154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:23.722154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:23.722154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:23.722154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:43:23.725297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 18/32] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:43:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:23.726275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aeafde-246c-4b14-8ee7-2ec50e07134f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:23.814588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 15/33] 10.222.0.67 () {60 vars in 1141 bytes} [Mon Sep 26 17:43:23 2022] POST /identity/v3/projects => generated 301 bytes in 439 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:43:23.816047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020cc0a0-dea3-4360-9217-7c06db58a98c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:23.816346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020cc0a0-dea3-4360-9217-7c06db58a98c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:23.816626 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020cc0a0-dea3-4360-9217-7c06db58a98c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:23.816824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:23.818772 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 16/34] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:23.823699 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020cc0a0-dea3-4360-9217-7c06db58a98c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:23.849095 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aeafde-246c-4b14-8ee7-2ec50e07134f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:23.894235 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25a7094-eeba-4314-a688-8f7de30e6543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:23.895161 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a7094-eeba-4314-a688-8f7de30e6543 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:23.895161 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a7094-eeba-4314-a688-8f7de30e6543 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:23.895450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25a7094-eeba-4314-a688-8f7de30e6543 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:23.896792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a7094-eeba-4314-a688-8f7de30e6543 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:23.898243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25a7094-eeba-4314-a688-8f7de30e6543 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:23.938409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020cc0a0-dea3-4360-9217-7c06db58a98c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:23.974767 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:23.988511 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ece8d2-9003-423a-bdeb-171f0308f04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:23.991935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ece8d2-9003-423a-bdeb-171f0308f04c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:23.991935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ece8d2-9003-423a-bdeb-171f0308f04c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:23.991935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ece8d2-9003-423a-bdeb-171f0308f04c None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:23.991935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ece8d2-9003-423a-bdeb-171f0308f04c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:23.991935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ece8d2-9003-423a-bdeb-171f0308f04c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:23.993091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ece8d2-9003-423a-bdeb-171f0308f04c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:24.007228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 17/35] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:24.007228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ece8d2-9003-423a-bdeb-171f0308f04c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:24.108422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ece8d2-9003-423a-bdeb-171f0308f04c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:24.114258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 19/36] 10.222.0.67 () {60 vars in 1131 bytes} [Mon Sep 26 17:43:23 2022] POST /identity/v3/roles => generated 219 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:43:24.116026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d6fe1-514d-44e7-bdb8-e911821c388a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:24.116316 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d6fe1-514d-44e7-bdb8-e911821c388a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:24.116595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329d6fe1-514d-44e7-bdb8-e911821c388a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:24.118033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 20/37] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:24.121017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae65fb-f4d3-4b08-afd0-83d837280352 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:24.121305 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae65fb-f4d3-4b08-afd0-83d837280352 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:24.121587 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae65fb-f4d3-4b08-afd0-83d837280352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:24.144372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-caae65fb-f4d3-4b08-afd0-83d837280352 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:24.146915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-caae65fb-f4d3-4b08-afd0-83d837280352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:24.152241 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274eeb12-6297-4b29-9ee2-9f757bba2bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:24.152883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eeb12-6297-4b29-9ee2-9f757bba2bc8 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:24.153169 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eeb12-6297-4b29-9ee2-9f757bba2bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:24.153449 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eeb12-6297-4b29-9ee2-9f757bba2bc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:24.154137 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274eeb12-6297-4b29-9ee2-9f757bba2bc8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:24.155569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274eeb12-6297-4b29-9ee2-9f757bba2bc8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:24.227138 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 18/38] 10.222.0.67 () {60 vars in 1131 bytes} [Mon Sep 26 17:43:24 2022] POST /identity/v3/roles => generated 225 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:43:24.228137 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274eeb12-6297-4b29-9ee2-9f757bba2bc8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:24.230115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 21/39] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:24.231715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0645358-6019-40ba-9776-6d2619826e40 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:24.232229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0645358-6019-40ba-9776-6d2619826e40 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:24.232663 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0645358-6019-40ba-9776-6d2619826e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:24.266660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d0645358-6019-40ba-9776-6d2619826e40 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:24.271933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d0645358-6019-40ba-9776-6d2619826e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:24.343636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:24.389273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:24.399799 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 22/40] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:24.413631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274eeb12-6297-4b29-9ee2-9f757bba2bc8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:24.451601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f9da4-44f9-4cb1-a640-d91c1208df72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:24.452294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f9da4-44f9-4cb1-a640-d91c1208df72 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:24.452569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f9da4-44f9-4cb1-a640-d91c1208df72 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:24.452851 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892f9da4-44f9-4cb1-a640-d91c1208df72 None admin] PATH_INFO: `/v3/users/e71ead94768141dc94d90c67625c3ac2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:24.454097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f9da4-44f9-4cb1-a640-d91c1208df72 None admin] RBAC: Authorizing `identity:get_user(user_id=e71ead94768141dc94d90c67625c3ac2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:24.455638 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892f9da4-44f9-4cb1-a640-d91c1208df72 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:24.457566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 19/41] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:24 2022] GET /identity/v3/users/e71ead94768141dc94d90c67625c3ac2 => generated 247 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:24.463322 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0645358-6019-40ba-9776-6d2619826e40 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:24.531160 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f9da4-44f9-4cb1-a640-d91c1208df72 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:24.578146 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0645358-6019-40ba-9776-6d2619826e40 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:24.659594 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892f9da4-44f9-4cb1-a640-d91c1208df72 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:24.677609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:24.696983 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f72861-18f8-4989-bbfe-cdc45b6b689a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:24.697697 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f72861-18f8-4989-bbfe-cdc45b6b689a None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:24.698204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f72861-18f8-4989-bbfe-cdc45b6b689a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:24.698634 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f72861-18f8-4989-bbfe-cdc45b6b689a None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:24.699563 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f72861-18f8-4989-bbfe-cdc45b6b689a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:24.701342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f72861-18f8-4989-bbfe-cdc45b6b689a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:24.720109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019be641-9fe8-4770-a97f-a5825a07b22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:24.720743 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019be641-9fe8-4770-a97f-a5825a07b22f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:24.721025 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019be641-9fe8-4770-a97f-a5825a07b22f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:24.721300 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019be641-9fe8-4770-a97f-a5825a07b22f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:24.722396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019be641-9fe8-4770-a97f-a5825a07b22f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:24.724035 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019be641-9fe8-4770-a97f-a5825a07b22f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:24.725875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 23/42] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:43:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:24.726691 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019be641-9fe8-4770-a97f-a5825a07b22f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:24.767738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 20/43] 10.222.0.67 () {60 vars in 1131 bytes} [Mon Sep 26 17:43:24 2022] POST /identity/v3/roles => generated 223 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:43:24.768548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f72861-18f8-4989-bbfe-cdc45b6b689a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:24.800789 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:24.844748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019be641-9fe8-4770-a97f-a5825a07b22f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:24.885256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f678f-18f2-47a6-8b4e-377fcc497dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:24.886289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f678f-18f2-47a6-8b4e-377fcc497dfa None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:24.886505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f72861-18f8-4989-bbfe-cdc45b6b689a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:24.886973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f678f-18f2-47a6-8b4e-377fcc497dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:24.887449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f678f-18f2-47a6-8b4e-377fcc497dfa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:24.888894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f678f-18f2-47a6-8b4e-377fcc497dfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:24.890805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f678f-18f2-47a6-8b4e-377fcc497dfa None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:24.892727 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 24/44] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:43:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:24.899156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f678f-18f2-47a6-8b4e-377fcc497dfa None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:24.925824 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7a3116-8227-408b-af5a-5e7eee052e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:24.926420 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a3116-8227-408b-af5a-5e7eee052e84 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:24.926711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a3116-8227-408b-af5a-5e7eee052e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:24.926990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7a3116-8227-408b-af5a-5e7eee052e84 None admin] PATH_INFO: `/v3/domains/default/users/e71ead94768141dc94d90c67625c3ac2/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:24.930176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a3116-8227-408b-af5a-5e7eee052e84 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e71ead94768141dc94d90c67625c3ac2, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:24.931687 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7a3116-8227-408b-af5a-5e7eee052e84 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:25.014217 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f678f-18f2-47a6-8b4e-377fcc497dfa None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:25.041439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 21/45] 10.222.0.67 () {58 vars in 1361 bytes} [Mon Sep 26 17:43:24 2022] PUT /identity/v3/domains/default/users/e71ead94768141dc94d90c67625c3ac2/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:43:25.047639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d4ed05-d10d-4f31-b10c-7e6d6133f33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:25.048290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4ed05-d10d-4f31-b10c-7e6d6133f33b None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:25.048571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4ed05-d10d-4f31-b10c-7e6d6133f33b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:25.048846 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d4ed05-d10d-4f31-b10c-7e6d6133f33b None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:25.049516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4ed05-d10d-4f31-b10c-7e6d6133f33b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:25.051012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d4ed05-d10d-4f31-b10c-7e6d6133f33b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:25.057407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:25.135140 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 25/46] 10.222.0.67 () {60 vars in 1141 bytes} [Mon Sep 26 17:43:24 2022] POST /identity/v3/projects => generated 312 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:43:27.225873 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5129672-ec38-4b13-aafc-53c7d83fbe78 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:27.226385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5129672-ec38-4b13-aafc-53c7d83fbe78 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:27.226466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5129672-ec38-4b13-aafc-53c7d83fbe78 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:27.227797 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 22/47] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:27.233135 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d6526-330a-4b93-8413-b3f6e99ca7ea None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:27.233542 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d6526-330a-4b93-8413-b3f6e99ca7ea None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:27.233651 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d6526-330a-4b93-8413-b3f6e99ca7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:27.251817 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a25d6526-330a-4b93-8413-b3f6e99ca7ea None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:27.253035 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a25d6526-330a-4b93-8413-b3f6e99ca7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:27.312974 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322643d0-b8de-4e4d-836e-468ddd5ac2ab None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:27.313268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322643d0-b8de-4e4d-836e-468ddd5ac2ab None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:27.313553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322643d0-b8de-4e4d-836e-468ddd5ac2ab None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:27.314896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 23/48] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:43:27.319702 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 26/49] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:27.320613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b08c7-6bd2-4f7e-a96b-e69def5650ed None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:27.320967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b08c7-6bd2-4f7e-a96b-e69def5650ed None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:27.321247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b08c7-6bd2-4f7e-a96b-e69def5650ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:27.339304 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-838b08c7-6bd2-4f7e-a96b-e69def5650ed None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:27.340861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-838b08c7-6bd2-4f7e-a96b-e69def5650ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:27.369832 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:27.408275 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 24/50] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:43:27.585717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d6526-330a-4b93-8413-b3f6e99ca7ea None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:27.681867 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25d6526-330a-4b93-8413-b3f6e99ca7ea None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:27.687271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b08c7-6bd2-4f7e-a96b-e69def5650ed None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:27.699186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:27.717646 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50aea3-0c70-4101-94d9-0c466fc3f23f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:27.718311 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50aea3-0c70-4101-94d9-0c466fc3f23f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:27.718588 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50aea3-0c70-4101-94d9-0c466fc3f23f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:27.718860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e50aea3-0c70-4101-94d9-0c466fc3f23f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:27.729601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e50aea3-0c70-4101-94d9-0c466fc3f23f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2e50aea3-0c70-4101-94d9-0c466fc3f23f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:27.734733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:27.735757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:27.736756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:27.736756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:27.736756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:27.736756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:27.736756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:27.736756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:27.736756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:27.736756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:27.736756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:27.736756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:27.736756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:27.736756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:43:27.736756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 27/51] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:43:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:43:27.741933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50aea3-0c70-4101-94d9-0c466fc3f23f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:27.791197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b08c7-6bd2-4f7e-a96b-e69def5650ed None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:27.825821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9d266-87d4-4795-978d-12676bb86e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:27.826464 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9d266-87d4-4795-978d-12676bb86e38 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:27.826751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9d266-87d4-4795-978d-12676bb86e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:27.827083 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9d266-87d4-4795-978d-12676bb86e38 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:27.828227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9d266-87d4-4795-978d-12676bb86e38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:27.829693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9d266-87d4-4795-978d-12676bb86e38 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:27.831295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 25/52] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:43:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:27.837575 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9d266-87d4-4795-978d-12676bb86e38 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:27.844595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e50aea3-0c70-4101-94d9-0c466fc3f23f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:27.871787 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:27.878730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3d171-9c61-4ea0-b11f-1d9f11896ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:27.879347 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3d171-9c61-4ea0-b11f-1d9f11896ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:27.879699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3d171-9c61-4ea0-b11f-1d9f11896ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:27.880024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b3d171-9c61-4ea0-b11f-1d9f11896ca8 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:27.881289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3d171-9c61-4ea0-b11f-1d9f11896ca8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:27.881630 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3d171-9c61-4ea0-b11f-1d9f11896ca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:27.883158 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b3d171-9c61-4ea0-b11f-1d9f11896ca8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:27.895380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 28/53] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:27.901810 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3d171-9c61-4ea0-b11f-1d9f11896ca8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:27.939627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9d266-87d4-4795-978d-12676bb86e38 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:27.944239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:27.974515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17797080-da0d-43ed-9a3b-8fc2526bf487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:27.975151 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17797080-da0d-43ed-9a3b-8fc2526bf487 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:27.975477 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17797080-da0d-43ed-9a3b-8fc2526bf487 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:27.975768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17797080-da0d-43ed-9a3b-8fc2526bf487 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:27.976427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17797080-da0d-43ed-9a3b-8fc2526bf487 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:27.977901 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17797080-da0d-43ed-9a3b-8fc2526bf487 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:28.006449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b3d171-9c61-4ea0-b11f-1d9f11896ca8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:28.041060 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d5718-d6e8-44b1-81c4-afbda1663f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:28.041693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d5718-d6e8-44b1-81c4-afbda1663f82 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:28.041976 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d5718-d6e8-44b1-81c4-afbda1663f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:28.042254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d5718-d6e8-44b1-81c4-afbda1663f82 None admin] PATH_INFO: `/v3/users/e71ead94768141dc94d90c67625c3ac2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:28.043387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d5718-d6e8-44b1-81c4-afbda1663f82 None admin] RBAC: Authorizing `identity:get_user(user_id=e71ead94768141dc94d90c67625c3ac2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:28.044868 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d5718-d6e8-44b1-81c4-afbda1663f82 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:28.046444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 29/54] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:27 2022] GET /identity/v3/users/e71ead94768141dc94d90c67625c3ac2 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:28.051770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d5718-d6e8-44b1-81c4-afbda1663f82 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:28.117288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 26/55] 10.222.0.67 () {60 vars in 1140 bytes} [Mon Sep 26 17:43:27 2022] POST /identity/v3/projects => generated 298 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:43:28.155769 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d5718-d6e8-44b1-81c4-afbda1663f82 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:28.164527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:28.197372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f2d31a-91e3-4d07-8ebd-5f140d7e4680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:28.199285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2d31a-91e3-4d07-8ebd-5f140d7e4680 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:28.199285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2d31a-91e3-4d07-8ebd-5f140d7e4680 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:28.199285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f2d31a-91e3-4d07-8ebd-5f140d7e4680 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:28.199705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2d31a-91e3-4d07-8ebd-5f140d7e4680 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:28.201157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f2d31a-91e3-4d07-8ebd-5f140d7e4680 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:28.203042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 30/56] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:43:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:28.209680 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17797080-da0d-43ed-9a3b-8fc2526bf487 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:28.299114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:28.314264 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17797080-da0d-43ed-9a3b-8fc2526bf487 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:28.348221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5a0d32-935a-4ed9-b03b-a9b112a03d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:28.349360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a0d32-935a-4ed9-b03b-a9b112a03d87 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:28.349747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a0d32-935a-4ed9-b03b-a9b112a03d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:28.350206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5a0d32-935a-4ed9-b03b-a9b112a03d87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:28.350928 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a0d32-935a-4ed9-b03b-a9b112a03d87 None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.domain.id=default, user.id=e71ead94768141dc94d90c67625c3ac2) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:28.351236 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a0d32-935a-4ed9-b03b-a9b112a03d87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:28.352704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5a0d32-935a-4ed9-b03b-a9b112a03d87 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:28.360823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 27/57] 10.222.0.67 () {56 vars in 1322 bytes} [Mon Sep 26 17:43:28 2022] GET /identity/v3/role_assignments?user.id=e71ead94768141dc94d90c67625c3ac2&scope.domain.id=default&role.id=07626899472b4e32adc0c71ce796a25d => generated 530 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:30.295342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5029101-c2ba-492c-b237-a5b950c1c0c9 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:30.295887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5029101-c2ba-492c-b237-a5b950c1c0c9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:30.295887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5029101-c2ba-492c-b237-a5b950c1c0c9 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:30.297174 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 31/58] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:30.302459 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c281488-7ff0-49e6-8e05-c0ef74b0efaf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:30.302951 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c281488-7ff0-49e6-8e05-c0ef74b0efaf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:30.303268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c281488-7ff0-49e6-8e05-c0ef74b0efaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:30.427192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8c281488-7ff0-49e6-8e05-c0ef74b0efaf None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:30.428768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8c281488-7ff0-49e6-8e05-c0ef74b0efaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:30.497028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 28/59] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:30.789055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5029101-c2ba-492c-b237-a5b950c1c0c9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:30.888952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5029101-c2ba-492c-b237-a5b950c1c0c9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:30.925773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1393e5e-80d3-4a8b-9bbd-42071577b786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:30.926396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1393e5e-80d3-4a8b-9bbd-42071577b786 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:30.926674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1393e5e-80d3-4a8b-9bbd-42071577b786 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:30.927477 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1393e5e-80d3-4a8b-9bbd-42071577b786 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:30.928704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1393e5e-80d3-4a8b-9bbd-42071577b786 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:30.930148 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1393e5e-80d3-4a8b-9bbd-42071577b786 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:30.931715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 32/60] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:43:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:30.937349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c281488-7ff0-49e6-8e05-c0ef74b0efaf None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:31.017753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:31.035805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c281488-7ff0-49e6-8e05-c0ef74b0efaf None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:31.069551 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ced057-1452-4802-a557-284d4e244f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:31.070095 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ced057-1452-4802-a557-284d4e244f41 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:31.070378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ced057-1452-4802-a557-284d4e244f41 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:31.070654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ced057-1452-4802-a557-284d4e244f41 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:31.071296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ced057-1452-4802-a557-284d4e244f41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:31.072766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ced057-1452-4802-a557-284d4e244f41 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:31.155117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 29/61] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:43:30 2022] POST /identity/v3/users => generated 275 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:43:34.433993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dd869-d8af-4ae0-b30c-40acafca181c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:34.434512 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dd869-d8af-4ae0-b30c-40acafca181c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:34.434584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057dd869-d8af-4ae0-b30c-40acafca181c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:34.436184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 33/62] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:34.446448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a314e29-5812-4263-afd0-2be1cf503119 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:34.447066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a314e29-5812-4263-afd0-2be1cf503119 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:34.447066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a314e29-5812-4263-afd0-2be1cf503119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:34.472954 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8a314e29-5812-4263-afd0-2be1cf503119 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:34.474790 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8a314e29-5812-4263-afd0-2be1cf503119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:34.556838 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6a2f7-f3d8-4332-a143-ee0ad2144d2c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:34.557171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6a2f7-f3d8-4332-a143-ee0ad2144d2c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:34.557385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc6a2f7-f3d8-4332-a143-ee0ad2144d2c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:34.558730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 34/63] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:43:34.563958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e173d70-420c-4a23-b05f-dcecb9c8f902 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:34.564483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e173d70-420c-4a23-b05f-dcecb9c8f902 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:34.568089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e173d70-420c-4a23-b05f-dcecb9c8f902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:34.582266 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 30/64] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:43:34.587812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2e173d70-420c-4a23-b05f-dcecb9c8f902 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:34.589388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2e173d70-420c-4a23-b05f-dcecb9c8f902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:34.629149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:34.670525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 35/65] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:34.710896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6b606-7f09-4d56-a3be-75b5a5615ebb None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:34.711160 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6b606-7f09-4d56-a3be-75b5a5615ebb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:34.711378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb6b606-7f09-4d56-a3be-75b5a5615ebb None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:34.713000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 31/66] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:34.729585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ee381-cbbc-4aca-8aac-e10ff8d25c93 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:34.730179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ee381-cbbc-4aca-8aac-e10ff8d25c93 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:34.730665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ee381-cbbc-4aca-8aac-e10ff8d25c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:34.751880 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-789ee381-cbbc-4aca-8aac-e10ff8d25c93 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:34.753394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-789ee381-cbbc-4aca-8aac-e10ff8d25c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:34.821386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 36/67] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:34.893533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6b606-7f09-4d56-a3be-75b5a5615ebb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:34.966259 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:35.016727 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb6b606-7f09-4d56-a3be-75b5a5615ebb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:35.056742 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ee381-cbbc-4aca-8aac-e10ff8d25c93 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:35.068118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b174a8-5653-4a3a-b13b-50d2cfa2c680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:35.068770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b174a8-5653-4a3a-b13b-50d2cfa2c680 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:35.069049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b174a8-5653-4a3a-b13b-50d2cfa2c680 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:35.069323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b174a8-5653-4a3a-b13b-50d2cfa2c680 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:35.080422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b174a8-5653-4a3a-b13b-50d2cfa2c680 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b1b174a8-5653-4a3a-b13b-50d2cfa2c680 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:35.086387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:35.087715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:35.088941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:35.088941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:35.088941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:35.088941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:35.088941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:35.088941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:35.088941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:35.088941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:35.088941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:35.088941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:35.088941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:43:35.088941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:43:35.088941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 32/68] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:35.093191 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b174a8-5653-4a3a-b13b-50d2cfa2c680 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:35.223443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b174a8-5653-4a3a-b13b-50d2cfa2c680 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:35.234229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ee381-cbbc-4aca-8aac-e10ff8d25c93 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:35.235861 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:35.253336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:35.262881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215f96f-f4bd-49a1-a5ef-968c51d6032f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:35.263567 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215f96f-f4bd-49a1-a5ef-968c51d6032f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:35.263886 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215f96f-f4bd-49a1-a5ef-968c51d6032f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:35.264163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215f96f-f4bd-49a1-a5ef-968c51d6032f None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:35.265106 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215f96f-f4bd-49a1-a5ef-968c51d6032f None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:35.265423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215f96f-f4bd-49a1-a5ef-968c51d6032f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:35.266801 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215f96f-f4bd-49a1-a5ef-968c51d6032f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:35.271173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02770cf-ec75-487a-b52c-476a10430039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:35.272038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02770cf-ec75-487a-b52c-476a10430039 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:35.274996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02770cf-ec75-487a-b52c-476a10430039 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:35.274996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02770cf-ec75-487a-b52c-476a10430039 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:35.275845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 33/69] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:43:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:35.276744 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e9845-29a1-46e4-a94e-8fcd52fad046 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:35.277034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e9845-29a1-46e4-a94e-8fcd52fad046 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:35.277311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80e9845-29a1-46e4-a94e-8fcd52fad046 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:35.278515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 34/70] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:35.283674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e9845-29a1-46e4-a94e-8fcd52fad046 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:35.288948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02770cf-ec75-487a-b52c-476a10430039 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b02770cf-ec75-487a-b52c-476a10430039 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:35.293930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:35.294753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:35.295492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:35.295492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:35.295492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:35.295492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:35.295492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:35.295492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:35.295492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:35.295492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:35.295492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:35.295492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:35.295492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:43:35.295492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:43:35.295492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 37/71] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:43:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:35.296627 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02770cf-ec75-487a-b52c-476a10430039 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:35.400494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02770cf-ec75-487a-b52c-476a10430039 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:35.410913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80e9845-29a1-46e4-a94e-8fcd52fad046 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:35.437866 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd0ee4-06f3-4fb0-b436-1f27d50e0f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:35.438590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd0ee4-06f3-4fb0-b436-1f27d50e0f57 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:35.438783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd0ee4-06f3-4fb0-b436-1f27d50e0f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:35.439056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd0ee4-06f3-4fb0-b436-1f27d50e0f57 None admin] PATH_INFO: `/v3/users/9b018c400b114999854f81cf97dcfca1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:35.449555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353a271-ba9b-40d0-a51e-48da89167b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:35.450263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353a271-ba9b-40d0-a51e-48da89167b7b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:35.450472 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353a271-ba9b-40d0-a51e-48da89167b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:35.450749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353a271-ba9b-40d0-a51e-48da89167b7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:35.453363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd0ee4-06f3-4fb0-b436-1f27d50e0f57 None admin] RBAC: Authorizing `identity:get_user(user_id=9b018c400b114999854f81cf97dcfca1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:35.454868 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd0ee4-06f3-4fb0-b436-1f27d50e0f57 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:35.456525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 38/72] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:35 2022] GET /identity/v3/users/9b018c400b114999854f81cf97dcfca1 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:35.457777 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6ea71-2d38-44e9-abe0-3267411a43f8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:35.458163 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6ea71-2d38-44e9-abe0-3267411a43f8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:35.458441 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6ea71-2d38-44e9-abe0-3267411a43f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:35.467071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c353a271-ba9b-40d0-a51e-48da89167b7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c353a271-ba9b-40d0-a51e-48da89167b7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:35.472664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:35.473715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:35.474481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:35.474481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:35.474481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:35.474481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:35.474481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:35.474481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:35.474481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:35.474481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:35.474481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:35.474481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:35.474481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:35.474481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:43:35.474481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 35/73] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:43:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:35.475088 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353a271-ba9b-40d0-a51e-48da89167b7b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:35.483000 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-eea6ea71-2d38-44e9-abe0-3267411a43f8 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:35.484295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eea6ea71-2d38-44e9-abe0-3267411a43f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:35.503281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:35.530599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:35.559219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 39/74] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:35.568926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6ea71-2d38-44e9-abe0-3267411a43f8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:35.590255 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c353a271-ba9b-40d0-a51e-48da89167b7b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:35.627816 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e67bc-5c9b-4ead-adaa-6ac64fec1534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:35.628430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e67bc-5c9b-4ead-adaa-6ac64fec1534 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:35.628712 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e67bc-5c9b-4ead-adaa-6ac64fec1534 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:35.628986 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31e67bc-5c9b-4ead-adaa-6ac64fec1534 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:35.629636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e67bc-5c9b-4ead-adaa-6ac64fec1534 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:35.630498 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e67bc-5c9b-4ead-adaa-6ac64fec1534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:35.631905 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31e67bc-5c9b-4ead-adaa-6ac64fec1534 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:35.640272 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 36/75] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:43:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:35.641040 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e67bc-5c9b-4ead-adaa-6ac64fec1534 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:35.701455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6ea71-2d38-44e9-abe0-3267411a43f8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:35.737125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45274844-c3f1-4835-bce3-b0f258de759e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:35.737644 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45274844-c3f1-4835-bce3-b0f258de759e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:35.737955 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45274844-c3f1-4835-bce3-b0f258de759e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:35.738232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45274844-c3f1-4835-bce3-b0f258de759e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:35.753389 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31e67bc-5c9b-4ead-adaa-6ac64fec1534 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:35.757034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:35.766043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:35.791558 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebadd6c-644b-41a4-a75a-ad671455204d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:35.792208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadd6c-644b-41a4-a75a-ad671455204d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:35.792489 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadd6c-644b-41a4-a75a-ad671455204d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:35.792769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadd6c-644b-41a4-a75a-ad671455204d None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:35.793424 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebadd6c-644b-41a4-a75a-ad671455204d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:35.793730 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebadd6c-644b-41a4-a75a-ad671455204d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:35.795127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebadd6c-644b-41a4-a75a-ad671455204d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:35.805178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 37/76] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:35.807042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6552688-56bf-4e92-b868-7aff6c68d071 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:35.807335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6552688-56bf-4e92-b868-7aff6c68d071 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:35.807614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6552688-56bf-4e92-b868-7aff6c68d071 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:35.808638 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45274844-c3f1-4835-bce3-b0f258de759e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:35.811763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 38/77] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:35.811763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6552688-56bf-4e92-b868-7aff6c68d071 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:35.811969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45274844-c3f1-4835-bce3-b0f258de759e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:35.910545 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 40/78] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:43:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:43:35.912098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45274844-c3f1-4835-bce3-b0f258de759e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:35.918428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6552688-56bf-4e92-b868-7aff6c68d071 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:35.952513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab56abe-536f-412e-9073-20e4413ac5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:35.953152 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab56abe-536f-412e-9073-20e4413ac5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:35.953432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab56abe-536f-412e-9073-20e4413ac5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:35.953708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab56abe-536f-412e-9073-20e4413ac5b9 None admin] PATH_INFO: `/v3/users/9b018c400b114999854f81cf97dcfca1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:35.954856 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab56abe-536f-412e-9073-20e4413ac5b9 None admin] RBAC: Authorizing `identity:get_user(user_id=9b018c400b114999854f81cf97dcfca1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:35.956324 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab56abe-536f-412e-9073-20e4413ac5b9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:35.958684 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 39/79] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:35 2022] GET /identity/v3/users/9b018c400b114999854f81cf97dcfca1 => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:35.959991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0855ef9-d03b-4840-ab66-344139083dc8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:35.960278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0855ef9-d03b-4840-ab66-344139083dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:35.960556 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0855ef9-d03b-4840-ab66-344139083dc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:35.968542 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:35.984825 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f0855ef9-d03b-4840-ab66-344139083dc8 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:35.986843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f0855ef9-d03b-4840-ab66-344139083dc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:36.021757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:36.025112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45274844-c3f1-4835-bce3-b0f258de759e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:36.055550 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 40/80] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:36.056430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0855ef9-d03b-4840-ab66-344139083dc8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:36.065008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756c361-8cf0-417a-bec7-94120f4264c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:36.065647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756c361-8cf0-417a-bec7-94120f4264c3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:36.065929 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756c361-8cf0-417a-bec7-94120f4264c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:36.066198 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756c361-8cf0-417a-bec7-94120f4264c3 None admin] PATH_INFO: `/v3/users/e71ead94768141dc94d90c67625c3ac2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:36.067876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756c361-8cf0-417a-bec7-94120f4264c3 None admin] RBAC: Authorizing `identity:get_user(user_id=e71ead94768141dc94d90c67625c3ac2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:36.070034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756c361-8cf0-417a-bec7-94120f4264c3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:36.071696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 41/81] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:35 2022] GET /identity/v3/users/e71ead94768141dc94d90c67625c3ac2 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:36.072412 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756c361-8cf0-417a-bec7-94120f4264c3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:36.167366 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0855ef9-d03b-4840-ab66-344139083dc8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:36.173588 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756c361-8cf0-417a-bec7-94120f4264c3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:36.193542 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:36.204759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacf21c-aca8-45c4-b5a3-93f7bc0b575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:36.205410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf21c-aca8-45c4-b5a3-93f7bc0b575f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:36.212965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf21c-aca8-45c4-b5a3-93f7bc0b575f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:36.212965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dacf21c-aca8-45c4-b5a3-93f7bc0b575f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:36.213315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b398f-ef26-463b-b809-35299868e5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:36.213315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b398f-ef26-463b-b809-35299868e5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:36.213315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b398f-ef26-463b-b809-35299868e5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:36.213315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5b398f-ef26-463b-b809-35299868e5c1 None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:36.213587 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b398f-ef26-463b-b809-35299868e5c1 None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:36.214758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5b398f-ef26-463b-b809-35299868e5c1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:36.216391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 42/82] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:36 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:36.217084 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b398f-ef26-463b-b809-35299868e5c1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:36.222528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dacf21c-aca8-45c4-b5a3-93f7bc0b575f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-7dacf21c-aca8-45c4-b5a3-93f7bc0b575f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:36.227618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:36.228569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:36.229467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:36.229467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:36.229467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:36.229467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:36.229467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:36.229467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:36.229467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:36.229467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:36.229467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:36.229467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:36.229467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:43:36.229467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:43:36.229467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 41/83] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:36.230214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacf21c-aca8-45c4-b5a3-93f7bc0b575f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:36.266715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:36.335748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5b398f-ef26-463b-b809-35299868e5c1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:36.346954 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dacf21c-aca8-45c4-b5a3-93f7bc0b575f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:36.370568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43489c-fc7f-430f-ad8c-58173172f7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:36.371187 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43489c-fc7f-430f-ad8c-58173172f7f2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:36.371466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43489c-fc7f-430f-ad8c-58173172f7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:36.371773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e43489c-fc7f-430f-ad8c-58173172f7f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:36.382813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6a663-3cf0-4b13-a671-64bb440d45d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:36.383429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6a663-3cf0-4b13-a671-64bb440d45d9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:36.383749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6a663-3cf0-4b13-a671-64bb440d45d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:36.384036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca6a663-3cf0-4b13-a671-64bb440d45d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:36.425996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:36.439749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43489c-fc7f-430f-ad8c-58173172f7f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:36.441257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e43489c-fc7f-430f-ad8c-58173172f7f2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:36.450919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6a663-3cf0-4b13-a671-64bb440d45d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:36.452426 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca6a663-3cf0-4b13-a671-64bb440d45d9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:36.480955 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:36.540386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 43/84] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:43:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:43:36.541239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43489c-fc7f-430f-ad8c-58173172f7f2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:36.561470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 42/85] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:43:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:43:36.561470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6a663-3cf0-4b13-a671-64bb440d45d9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:36.630402 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:36.645351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e43489c-fc7f-430f-ad8c-58173172f7f2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:36.656182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca6a663-3cf0-4b13-a671-64bb440d45d9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:36.680863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4ceac-5abf-4296-9a7d-db1aa006eaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:36.681531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4ceac-5abf-4296-9a7d-db1aa006eaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:36.681815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4ceac-5abf-4296-9a7d-db1aa006eaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:36.682085 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4ceac-5abf-4296-9a7d-db1aa006eaf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:36.682810 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4ceac-5abf-4296-9a7d-db1aa006eaf0 None admin] RBAC: Adding query filter params (role.id=1c983c7e32e44c61be3996c28399e617, scope.project.id=25808aef0f69482dbf49592f737cff01, user.id=9b018c400b114999854f81cf97dcfca1) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:36.683113 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4ceac-5abf-4296-9a7d-db1aa006eaf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:36.684592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b4ceac-5abf-4296-9a7d-db1aa006eaf0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:36.684762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:36.689983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 44/86] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:43:36 2022] GET /identity/v3/role_assignments?user.id=9b018c400b114999854f81cf97dcfca1&scope.project.id=25808aef0f69482dbf49592f737cff01&role.id=1c983c7e32e44c61be3996c28399e617 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:36.690701 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2a6fa-9fd7-4726-8cc3-5d5792eb447f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:36.690833 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4ceac-5abf-4296-9a7d-db1aa006eaf0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:36.691319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2a6fa-9fd7-4726-8cc3-5d5792eb447f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:36.691929 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2a6fa-9fd7-4726-8cc3-5d5792eb447f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:36.691929 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d2a6fa-9fd7-4726-8cc3-5d5792eb447f None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:36.693251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2a6fa-9fd7-4726-8cc3-5d5792eb447f None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:36.693555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2a6fa-9fd7-4726-8cc3-5d5792eb447f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:36.694973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d2a6fa-9fd7-4726-8cc3-5d5792eb447f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:36.704662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 43/87] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:43:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:36.705664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2a6fa-9fd7-4726-8cc3-5d5792eb447f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:36.821320 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d2a6fa-9fd7-4726-8cc3-5d5792eb447f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:36.836767 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b4ceac-5abf-4296-9a7d-db1aa006eaf0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:36.860897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fbf8d-8c04-481b-96a4-d81195fa6401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:36.861538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fbf8d-8c04-481b-96a4-d81195fa6401 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:36.861817 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fbf8d-8c04-481b-96a4-d81195fa6401 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:36.862088 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004fbf8d-8c04-481b-96a4-d81195fa6401 None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:36.863235 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fbf8d-8c04-481b-96a4-d81195fa6401 None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:36.864770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004fbf8d-8c04-481b-96a4-d81195fa6401 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:36.867757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 44/88] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:36 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:36.867757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fbf8d-8c04-481b-96a4-d81195fa6401 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:36.882153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80947887-35d2-461d-bba0-3e265317a736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:36.883098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80947887-35d2-461d-bba0-3e265317a736 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:36.883385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80947887-35d2-461d-bba0-3e265317a736 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:36.883683 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80947887-35d2-461d-bba0-3e265317a736 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:36.885406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80947887-35d2-461d-bba0-3e265317a736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:36.887167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80947887-35d2-461d-bba0-3e265317a736 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:36.889980 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 45/89] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:43:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:36.890703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80947887-35d2-461d-bba0-3e265317a736 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:36.911195 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:36.936787 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:36.986893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004fbf8d-8c04-481b-96a4-d81195fa6401 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:36.998494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80947887-35d2-461d-bba0-3e265317a736 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:37.025951 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6628f1e-51fb-48ed-82a3-2900f0647d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:37.026734 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6628f1e-51fb-48ed-82a3-2900f0647d3e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:37.027012 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6628f1e-51fb-48ed-82a3-2900f0647d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:37.027292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6628f1e-51fb-48ed-82a3-2900f0647d3e None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:37.028429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6628f1e-51fb-48ed-82a3-2900f0647d3e None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:37.030113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6628f1e-51fb-48ed-82a3-2900f0647d3e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:37.031914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 45/90] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:36 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:37.032601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6628f1e-51fb-48ed-82a3-2900f0647d3e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:37.039975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f1d6b-ed6f-4650-9f60-b701348e2ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:37.040627 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f1d6b-ed6f-4650-9f60-b701348e2ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:37.040911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f1d6b-ed6f-4650-9f60-b701348e2ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:37.041188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f1d6b-ed6f-4650-9f60-b701348e2ba1 None admin] PATH_INFO: `/v3/users/9b018c400b114999854f81cf97dcfca1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:37.042344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f1d6b-ed6f-4650-9f60-b701348e2ba1 None admin] RBAC: Authorizing `identity:get_user(user_id=9b018c400b114999854f81cf97dcfca1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:37.043840 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f1d6b-ed6f-4650-9f60-b701348e2ba1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:37.049851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 46/91] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:36 2022] GET /identity/v3/users/9b018c400b114999854f81cf97dcfca1 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:37.049851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f1d6b-ed6f-4650-9f60-b701348e2ba1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:37.135272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:37.146749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6628f1e-51fb-48ed-82a3-2900f0647d3e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:37.152466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f1d6b-ed6f-4650-9f60-b701348e2ba1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:37.175887 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:37.185200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5776f25-f305-48aa-93e3-e33789cf4762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:37.185882 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5776f25-f305-48aa-93e3-e33789cf4762 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:37.186176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5776f25-f305-48aa-93e3-e33789cf4762 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:37.186457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5776f25-f305-48aa-93e3-e33789cf4762 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:37.187174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5776f25-f305-48aa-93e3-e33789cf4762 None admin] RBAC: Adding query filter params (role.id=30bb990dfe5c45fbbfed21c08c7859b4, scope.project.id=25808aef0f69482dbf49592f737cff01, user.id=9b018c400b114999854f81cf97dcfca1) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:37.187488 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5776f25-f305-48aa-93e3-e33789cf4762 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:37.188960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5776f25-f305-48aa-93e3-e33789cf4762 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:37.191501 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290125a-9f9e-484f-91cb-fc9aa3ecf77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:37.193026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290125a-9f9e-484f-91cb-fc9aa3ecf77b None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:37.193499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290125a-9f9e-484f-91cb-fc9aa3ecf77b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:37.193842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d290125a-9f9e-484f-91cb-fc9aa3ecf77b None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:37.194597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d290125a-9f9e-484f-91cb-fc9aa3ecf77b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:37.195600 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 46/92] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:43:37 2022] GET /identity/v3/role_assignments?user.id=9b018c400b114999854f81cf97dcfca1&scope.project.id=25808aef0f69482dbf49592f737cff01&role.id=30bb990dfe5c45fbbfed21c08c7859b4 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:37.196428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5776f25-f305-48aa-93e3-e33789cf4762 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:37.207696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d290125a-9f9e-484f-91cb-fc9aa3ecf77b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:37.309090 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 47/93] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:43:37 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:43:37.310108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290125a-9f9e-484f-91cb-fc9aa3ecf77b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:37.331325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5776f25-f305-48aa-93e3-e33789cf4762 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:37.375277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05803c9-6b47-4d6e-8a5c-dd008f6a031a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:37.376615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05803c9-6b47-4d6e-8a5c-dd008f6a031a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:37.377369 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05803c9-6b47-4d6e-8a5c-dd008f6a031a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:37.377816 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05803c9-6b47-4d6e-8a5c-dd008f6a031a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:37.378713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05803c9-6b47-4d6e-8a5c-dd008f6a031a None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=25808aef0f69482dbf49592f737cff01, user.id=e71ead94768141dc94d90c67625c3ac2) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:37.379185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05803c9-6b47-4d6e-8a5c-dd008f6a031a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:37.381435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05803c9-6b47-4d6e-8a5c-dd008f6a031a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:37.389626 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 47/94] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:43:37 2022] GET /identity/v3/role_assignments?user.id=e71ead94768141dc94d90c67625c3ac2&scope.project.id=25808aef0f69482dbf49592f737cff01&role.id=07626899472b4e32adc0c71ce796a25d => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:37.455679 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d290125a-9f9e-484f-91cb-fc9aa3ecf77b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:37.500815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822780b-2c2d-400a-9559-3ac54bff973b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:37.501457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822780b-2c2d-400a-9559-3ac54bff973b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:37.501735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822780b-2c2d-400a-9559-3ac54bff973b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:37.502008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4822780b-2c2d-400a-9559-3ac54bff973b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:37.522103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:37.651621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822780b-2c2d-400a-9559-3ac54bff973b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:37.653082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4822780b-2c2d-400a-9559-3ac54bff973b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:37.774276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 48/95] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:43:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:43:37.779378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05803c9-6b47-4d6e-8a5c-dd008f6a031a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:37.859064 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:37.915557 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05803c9-6b47-4d6e-8a5c-dd008f6a031a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:37.957070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce691c-676a-48a9-85fc-ba7d184b2f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:37.957728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce691c-676a-48a9-85fc-ba7d184b2f5d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:37.958624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce691c-676a-48a9-85fc-ba7d184b2f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:37.958908 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce691c-676a-48a9-85fc-ba7d184b2f5d None admin] PATH_INFO: `/v3/projects/22c186e05ced4a909333963ed82fbddb` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:37.960292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce691c-676a-48a9-85fc-ba7d184b2f5d None admin] RBAC: Authorizing `identity:get_project(project_id=22c186e05ced4a909333963ed82fbddb)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:37.961760 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce691c-676a-48a9-85fc-ba7d184b2f5d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:37.963418 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 48/96] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:37 2022] GET /identity/v3/projects/22c186e05ced4a909333963ed82fbddb => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:37.969391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822780b-2c2d-400a-9559-3ac54bff973b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:38.020042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:38.081934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4822780b-2c2d-400a-9559-3ac54bff973b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:38.123173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8dd179-741f-44c0-8ffe-abc8fb40a239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:38.123826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dd179-741f-44c0-8ffe-abc8fb40a239 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:38.124115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dd179-741f-44c0-8ffe-abc8fb40a239 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:38.124393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8dd179-741f-44c0-8ffe-abc8fb40a239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:38.125130 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8dd179-741f-44c0-8ffe-abc8fb40a239 None admin] RBAC: Adding query filter params (role.id=1c983c7e32e44c61be3996c28399e617, scope.project.id=22c186e05ced4a909333963ed82fbddb, user.id=9b018c400b114999854f81cf97dcfca1) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:38.125444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8dd179-741f-44c0-8ffe-abc8fb40a239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:38.126911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8dd179-741f-44c0-8ffe-abc8fb40a239 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:38.133326 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 49/97] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:43:37 2022] GET /identity/v3/role_assignments?user.id=9b018c400b114999854f81cf97dcfca1&scope.project.id=22c186e05ced4a909333963ed82fbddb&role.id=1c983c7e32e44c61be3996c28399e617 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:39.480236 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fa6d6-688b-4946-a5c7-3da16542113c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:39.480747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fa6d6-688b-4946-a5c7-3da16542113c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:39.480920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fa6d6-688b-4946-a5c7-3da16542113c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:39.482327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 49/98] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:39.487880 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaad0e-f564-4958-9d17-ab423aeb1bcd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:39.488677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaad0e-f564-4958-9d17-ab423aeb1bcd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:39.489186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebaad0e-f564-4958-9d17-ab423aeb1bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:39.516641 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4ebaad0e-f564-4958-9d17-ab423aeb1bcd None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:39.518187 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4ebaad0e-f564-4958-9d17-ab423aeb1bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:39.600975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 50/99] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:40.008661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb39303-e557-48ad-9cdf-ff9a41230a02 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:40.009274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb39303-e557-48ad-9cdf-ff9a41230a02 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:40.009838 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb39303-e557-48ad-9cdf-ff9a41230a02 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:40.011380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 50/100] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:40.017243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7fec0-29cb-4833-9498-744ddad6c218 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:40.017766 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7fec0-29cb-4833-9498-744ddad6c218 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:40.018233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb7fec0-29cb-4833-9498-744ddad6c218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:40.041684 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-bbb7fec0-29cb-4833-9498-744ddad6c218 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:40.043213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bbb7fec0-29cb-4833-9498-744ddad6c218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:40.091948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:40.145153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 51/101] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:43:40.227743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb39303-e557-48ad-9cdf-ff9a41230a02 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:40.363926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb39303-e557-48ad-9cdf-ff9a41230a02 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:40.396676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:40.408298 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfe4c5-22ca-427e-a355-351d99463ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:40.408946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfe4c5-22ca-427e-a355-351d99463ffa None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:40.409233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfe4c5-22ca-427e-a355-351d99463ffa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:40.409513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdfe4c5-22ca-427e-a355-351d99463ffa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:40.426731 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdfe4c5-22ca-427e-a355-351d99463ffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6fdfe4c5-22ca-427e-a355-351d99463ffa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:40.431749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:40.432546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:40.433305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:40.433305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:40.433305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:40.433305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:40.433305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:40.433305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:40.433305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:40.433305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:40.433305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:40.433305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:40.433305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:43:40.433305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:43:40.433305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 51/102] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:40.439434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7fec0-29cb-4833-9498-744ddad6c218 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:40.484790 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfe4c5-22ca-427e-a355-351d99463ffa None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:40.646327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb7fec0-29cb-4833-9498-744ddad6c218 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:40.668896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdfe4c5-22ca-427e-a355-351d99463ffa None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:40.701665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6182f-9ada-4c3e-9e60-de22ffd7c23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:40.702298 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6182f-9ada-4c3e-9e60-de22ffd7c23b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:40.702585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6182f-9ada-4c3e-9e60-de22ffd7c23b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:40.702864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6182f-9ada-4c3e-9e60-de22ffd7c23b None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:40.703543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6182f-9ada-4c3e-9e60-de22ffd7c23b None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:40.703884 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6182f-9ada-4c3e-9e60-de22ffd7c23b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:40.705297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6182f-9ada-4c3e-9e60-de22ffd7c23b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:40.717658 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 52/103] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:43:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:40.718779 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e1ab4-ca06-477f-b7d7-83e12c4ea27b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:40.719072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e1ab4-ca06-477f-b7d7-83e12c4ea27b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:40.719354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e1ab4-ca06-477f-b7d7-83e12c4ea27b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:40.720749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 53/104] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:40.730150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e1ab4-ca06-477f-b7d7-83e12c4ea27b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:40.744817 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1313023-e6ab-4712-9356-da3685da6065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:40.745483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1313023-e6ab-4712-9356-da3685da6065 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:40.745769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1313023-e6ab-4712-9356-da3685da6065 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:40.746050 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1313023-e6ab-4712-9356-da3685da6065 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:40.764726 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1313023-e6ab-4712-9356-da3685da6065 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c1313023-e6ab-4712-9356-da3685da6065 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:40.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:40.771852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:40.772603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:40.772603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:40.772603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:40.772603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:40.772603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:40.772603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:40.772603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:40.772603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:40.772603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:40.772603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:40.772603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:43:40.772603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:43:40.772603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 52/105] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:43:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:43:40.773672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63eb0d-f234-4cb2-a842-2a4939033529 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:40.773836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63eb0d-f234-4cb2-a842-2a4939033529 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:40.774078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c63eb0d-f234-4cb2-a842-2a4939033529 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:40.804046 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:40.805678 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4c63eb0d-f234-4cb2-a842-2a4939033529 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:40.806906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4c63eb0d-f234-4cb2-a842-2a4939033529 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:40.907753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:40.930433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e1ab4-ca06-477f-b7d7-83e12c4ea27b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:40.933658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 53/106] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:40.934621 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63eb0d-f234-4cb2-a842-2a4939033529 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:41.023698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacd956-68f5-465d-bf88-3635364826a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:41.024351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacd956-68f5-465d-bf88-3635364826a7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:41.026380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacd956-68f5-465d-bf88-3635364826a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:41.027389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacd956-68f5-465d-bf88-3635364826a7 None admin] PATH_INFO: `/v3/users/9b018c400b114999854f81cf97dcfca1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:41.029496 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacd956-68f5-465d-bf88-3635364826a7 None admin] RBAC: Authorizing `identity:get_user(user_id=9b018c400b114999854f81cf97dcfca1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:41.031193 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aacd956-68f5-465d-bf88-3635364826a7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:41.038562 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 54/107] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:40 2022] GET /identity/v3/users/9b018c400b114999854f81cf97dcfca1 => generated 275 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:41.041251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacd956-68f5-465d-bf88-3635364826a7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:41.125034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c63eb0d-f234-4cb2-a842-2a4939033529 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:41.158351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:41.165447 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacd956-68f5-465d-bf88-3635364826a7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:41.173197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992044e4-bcb6-49e9-a357-5efb25675731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:41.173703 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992044e4-bcb6-49e9-a357-5efb25675731 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:41.173994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992044e4-bcb6-49e9-a357-5efb25675731 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:41.174269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992044e4-bcb6-49e9-a357-5efb25675731 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:41.174938 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992044e4-bcb6-49e9-a357-5efb25675731 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:41.175240 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992044e4-bcb6-49e9-a357-5efb25675731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:41.177650 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992044e4-bcb6-49e9-a357-5efb25675731 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:41.196595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 54/108] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:43:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:41.197910 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dada9ff-442b-467b-b75c-02d9ee6d6fe3 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:41.198251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dada9ff-442b-467b-b75c-02d9ee6d6fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:41.198592 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dada9ff-442b-467b-b75c-02d9ee6d6fe3 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:41.200002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 55/109] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:41.203297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30082204-77b7-4c23-818b-27253c693e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:41.204182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30082204-77b7-4c23-818b-27253c693e12 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:41.204648 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30082204-77b7-4c23-818b-27253c693e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:41.204798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dada9ff-442b-467b-b75c-02d9ee6d6fe3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:41.205440 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30082204-77b7-4c23-818b-27253c693e12 None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:41.206771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30082204-77b7-4c23-818b-27253c693e12 None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:41.208504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30082204-77b7-4c23-818b-27253c693e12 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:41.211689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 55/110] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:41 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:41.213060 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a166581-1c3b-4ed1-8931-5cd4c37ac425 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:41.213549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a166581-1c3b-4ed1-8931-5cd4c37ac425 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:41.214001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a166581-1c3b-4ed1-8931-5cd4c37ac425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:41.234413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5a166581-1c3b-4ed1-8931-5cd4c37ac425 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:41.236082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5a166581-1c3b-4ed1-8931-5cd4c37ac425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:41.281212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:41.330347 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 56/111] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:41.331446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a166581-1c3b-4ed1-8931-5cd4c37ac425 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:41.367815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dada9ff-442b-467b-b75c-02d9ee6d6fe3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:41.407012 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826b7de-3ec1-46aa-a83e-16b34b9a85d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:41.408100 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826b7de-3ec1-46aa-a83e-16b34b9a85d0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:41.408663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826b7de-3ec1-46aa-a83e-16b34b9a85d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:41.409135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5826b7de-3ec1-46aa-a83e-16b34b9a85d0 None admin] PATH_INFO: `/v3/users/9b018c400b114999854f81cf97dcfca1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:41.410481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826b7de-3ec1-46aa-a83e-16b34b9a85d0 None admin] RBAC: Authorizing `identity:get_user(user_id=9b018c400b114999854f81cf97dcfca1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:41.412690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5826b7de-3ec1-46aa-a83e-16b34b9a85d0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:41.414662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 56/112] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:41 2022] GET /identity/v3/users/9b018c400b114999854f81cf97dcfca1 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:41.415675 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826b7de-3ec1-46aa-a83e-16b34b9a85d0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:41.432044 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:41.438566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a166581-1c3b-4ed1-8931-5cd4c37ac425 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:41.474783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb9786-aebb-4a33-ab32-5a36748bacd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:41.475707 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb9786-aebb-4a33-ab32-5a36748bacd0 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:41.476163 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb9786-aebb-4a33-ab32-5a36748bacd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:41.476582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb9786-aebb-4a33-ab32-5a36748bacd0 None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01/users/9b018c400b114999854f81cf97dcfca1/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:41.478687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb9786-aebb-4a33-ab32-5a36748bacd0 None admin] RBAC: Authorizing `identity:create_grant(project_id=25808aef0f69482dbf49592f737cff01, user_id=9b018c400b114999854f81cf97dcfca1, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:41.480327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb9786-aebb-4a33-ab32-5a36748bacd0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:41.530647 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5826b7de-3ec1-46aa-a83e-16b34b9a85d0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:41.543071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 57/113] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:43:41 2022] PUT /identity/v3/projects/25808aef0f69482dbf49592f737cff01/users/9b018c400b114999854f81cf97dcfca1/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:43:41.543291 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:41.543416 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb9786-aebb-4a33-ab32-5a36748bacd0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:41.619408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fec1b-f007-486b-a1ca-cfaa30c472bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:41.620113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fec1b-f007-486b-a1ca-cfaa30c472bf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:41.620401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fec1b-f007-486b-a1ca-cfaa30c472bf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:41.620682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fec1b-f007-486b-a1ca-cfaa30c472bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:41.633640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03fec1b-f007-486b-a1ca-cfaa30c472bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e03fec1b-f007-486b-a1ca-cfaa30c472bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:41.640520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:41.641408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:41.642555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:41.642555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:41.642555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:41.642555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:41.642555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:41.642555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:41.642555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:41.642555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:41.642555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:41.642555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:41.642555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:41.642555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:43:41.642555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 57/114] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:43:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:41.643420 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585883e-35c5-4842-aedb-3c721c20a66c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:41.643420 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585883e-35c5-4842-aedb-3c721c20a66c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:41.643420 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1585883e-35c5-4842-aedb-3c721c20a66c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:41.645832 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 58/115] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:41.646069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012a384-6acf-42ce-9785-2500385d3429 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:41.646295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012a384-6acf-42ce-9785-2500385d3429 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:41.646571 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012a384-6acf-42ce-9785-2500385d3429 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:41.648298 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 59/116] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:41.649151 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012a384-6acf-42ce-9785-2500385d3429 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:41.665350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb9786-aebb-4a33-ab32-5a36748bacd0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:41.703640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487cd9e-ac87-40cf-ae06-84bc3a5c8fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:41.704323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487cd9e-ac87-40cf-ae06-84bc3a5c8fc9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:41.704612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487cd9e-ac87-40cf-ae06-84bc3a5c8fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:41.704892 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487cd9e-ac87-40cf-ae06-84bc3a5c8fc9 None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:41.706065 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487cd9e-ac87-40cf-ae06-84bc3a5c8fc9 None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:41.707557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c487cd9e-ac87-40cf-ae06-84bc3a5c8fc9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:41.710439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 58/117] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:41 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:41.711596 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4e030-a839-41b4-a47c-fa0a3eab08bb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:41.711913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4e030-a839-41b4-a47c-fa0a3eab08bb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:41.712200 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d4e030-a839-41b4-a47c-fa0a3eab08bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:41.737187 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-56d4e030-a839-41b4-a47c-fa0a3eab08bb None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:41.739837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-56d4e030-a839-41b4-a47c-fa0a3eab08bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:41.765060 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:41.765547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012a384-6acf-42ce-9785-2500385d3429 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:41.809528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0bf04-670c-4649-9aef-1323a21f88b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:41.810456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0bf04-670c-4649-9aef-1323a21f88b8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:41.810908 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0bf04-670c-4649-9aef-1323a21f88b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:41.811382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b0bf04-670c-4649-9aef-1323a21f88b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:41.812232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0bf04-670c-4649-9aef-1323a21f88b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:41.812706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0bf04-670c-4649-9aef-1323a21f88b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:41.814966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b0bf04-670c-4649-9aef-1323a21f88b8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:41.817975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 59/118] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:41.819317 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3c3b1-5d46-42da-b4da-13c71d680d4a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:41.819608 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3c3b1-5d46-42da-b4da-13c71d680d4a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:41.819912 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3c3b1-5d46-42da-b4da-13c71d680d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:41.825525 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 60/119] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:41.825525 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0bf04-670c-4649-9aef-1323a21f88b8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:41.847693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-19e3c3b1-5d46-42da-b4da-13c71d680d4a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:41.849862 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-19e3c3b1-5d46-42da-b4da-13c71d680d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:41.861334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:41.926351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 60/120] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:41.928119 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3c3b1-5d46-42da-b4da-13c71d680d4a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:42.013892 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b0bf04-670c-4649-9aef-1323a21f88b8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:42.032132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:42.069120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95f526-2170-4baa-bf96-b6bbfec398f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:42.069758 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95f526-2170-4baa-bf96-b6bbfec398f8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:42.070038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95f526-2170-4baa-bf96-b6bbfec398f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:42.070313 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95f526-2170-4baa-bf96-b6bbfec398f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:42.071428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95f526-2170-4baa-bf96-b6bbfec398f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:42.072896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95f526-2170-4baa-bf96-b6bbfec398f8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:42.073574 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3c3b1-5d46-42da-b4da-13c71d680d4a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:42.074567 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 61/121] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:43:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:42.075321 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95f526-2170-4baa-bf96-b6bbfec398f8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:42.122383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f53bf-a2e6-45c3-9c75-3f11127757ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:42.123521 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f53bf-a2e6-45c3-9c75-3f11127757ba None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:42.123844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f53bf-a2e6-45c3-9c75-3f11127757ba None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:42.124126 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4f53bf-a2e6-45c3-9c75-3f11127757ba None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01/users/9b018c400b114999854f81cf97dcfca1/roles/30bb990dfe5c45fbbfed21c08c7859b4` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:42.127019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f53bf-a2e6-45c3-9c75-3f11127757ba None admin] RBAC: Authorizing `identity:create_grant(project_id=25808aef0f69482dbf49592f737cff01, user_id=9b018c400b114999854f81cf97dcfca1, role_id=30bb990dfe5c45fbbfed21c08c7859b4)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:42.129272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4f53bf-a2e6-45c3-9c75-3f11127757ba None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:42.178611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:42.185071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95f526-2170-4baa-bf96-b6bbfec398f8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:42.219244 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5fb45-666d-4a17-838b-9790c69dc8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:42.219931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5fb45-666d-4a17-838b-9790c69dc8be None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:42.220212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5fb45-666d-4a17-838b-9790c69dc8be None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:42.220487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5fb45-666d-4a17-838b-9790c69dc8be None admin] PATH_INFO: `/v3/users/e71ead94768141dc94d90c67625c3ac2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:42.221612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5fb45-666d-4a17-838b-9790c69dc8be None admin] RBAC: Authorizing `identity:get_user(user_id=e71ead94768141dc94d90c67625c3ac2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:42.223284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db5fb45-666d-4a17-838b-9790c69dc8be None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:42.226962 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 62/122] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:42 2022] GET /identity/v3/users/e71ead94768141dc94d90c67625c3ac2 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:42.228181 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5fb45-666d-4a17-838b-9790c69dc8be None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:42.241703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 61/123] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:43:41 2022] PUT /identity/v3/projects/25808aef0f69482dbf49592f737cff01/users/9b018c400b114999854f81cf97dcfca1/roles/30bb990dfe5c45fbbfed21c08c7859b4 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:43:42.248728 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f53bf-a2e6-45c3-9c75-3f11127757ba None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:42.364099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db5fb45-666d-4a17-838b-9790c69dc8be None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:42.379208 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4f53bf-a2e6-45c3-9c75-3f11127757ba None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:42.405593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335494dd-8dca-472a-ad4d-4bc7928b4488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:42.406219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335494dd-8dca-472a-ad4d-4bc7928b4488 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:42.406502 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335494dd-8dca-472a-ad4d-4bc7928b4488 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:42.406785 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335494dd-8dca-472a-ad4d-4bc7928b4488 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:42.407449 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335494dd-8dca-472a-ad4d-4bc7928b4488 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:42.409039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335494dd-8dca-472a-ad4d-4bc7928b4488 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:42.422517 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7df99d-f625-4357-b869-3e97afb820f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:42.423179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7df99d-f625-4357-b869-3e97afb820f4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:42.423457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7df99d-f625-4357-b869-3e97afb820f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:42.423926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7df99d-f625-4357-b869-3e97afb820f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:42.443049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7df99d-f625-4357-b869-3e97afb820f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:42.443049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2b7df99d-f625-4357-b869-3e97afb820f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:43:42.443049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:42.443049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:42.443049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:42.443049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:42.443049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:42.443049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:42.443049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:42.443049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:42.443049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:42.443049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:42.443049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:42.443049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:42.444072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:42.444941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:42.444941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:42.444941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:42.444941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:42.444941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:42.444941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:42.444941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:42.444941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:42.444941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:42.444941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:42.444941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:42.444941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:42.444941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:43:42.444941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:43:42.444941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 62/124] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:42.445635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7df99d-f625-4357-b869-3e97afb820f4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:42.449066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:42.501228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 63/125] 10.222.0.67 () {60 vars in 1141 bytes} [Mon Sep 26 17:43:42 2022] POST /identity/v3/projects => generated 302 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:43:42.508254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335494dd-8dca-472a-ad4d-4bc7928b4488 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:42.550939 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:42.646360 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7df99d-f625-4357-b869-3e97afb820f4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:42.717648 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa546e5c-6d5e-44e4-9741-f0df4c83568e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:42.719037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa546e5c-6d5e-44e4-9741-f0df4c83568e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:42.719579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa546e5c-6d5e-44e4-9741-f0df4c83568e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:42.720186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa546e5c-6d5e-44e4-9741-f0df4c83568e None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:42.722357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa546e5c-6d5e-44e4-9741-f0df4c83568e None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:42.722979 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335494dd-8dca-472a-ad4d-4bc7928b4488 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:42.725364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa546e5c-6d5e-44e4-9741-f0df4c83568e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:42.727369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 63/126] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:42 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:42.732571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa546e5c-6d5e-44e4-9741-f0df4c83568e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:42.765075 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4fa7b-f3d2-43e8-a5cb-9a19aaf0bf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:42.765987 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4fa7b-f3d2-43e8-a5cb-9a19aaf0bf1d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:42.766277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4fa7b-f3d2-43e8-a5cb-9a19aaf0bf1d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:42.766556 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d4fa7b-f3d2-43e8-a5cb-9a19aaf0bf1d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:42.787013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d4fa7b-f3d2-43e8-a5cb-9a19aaf0bf1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e4d4fa7b-f3d2-43e8-a5cb-9a19aaf0bf1d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:42.793125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:42.794052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:42.794821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:42.794821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:42.794821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:42.794821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:42.794821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:42.794821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:42.794821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:42.794821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:42.794821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:42.794821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:42.794821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:43:42.794821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:43:42.794821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 64/127] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:42.795430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4fa7b-f3d2-43e8-a5cb-9a19aaf0bf1d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:42.832961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:42.857353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa546e5c-6d5e-44e4-9741-f0df4c83568e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:42.900544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f24f34-2639-456c-8d9b-ea2b0da6a633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:42.901550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f24f34-2639-456c-8d9b-ea2b0da6a633 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:42.902013 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f24f34-2639-456c-8d9b-ea2b0da6a633 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:42.902228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:42.903052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f24f34-2639-456c-8d9b-ea2b0da6a633 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:42.904020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f24f34-2639-456c-8d9b-ea2b0da6a633 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:42.904519 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f24f34-2639-456c-8d9b-ea2b0da6a633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:42.906090 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f24f34-2639-456c-8d9b-ea2b0da6a633 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:42.916299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 64/128] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:43:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:42.917386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f24f34-2639-456c-8d9b-ea2b0da6a633 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:42.933811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d4fa7b-f3d2-43e8-a5cb-9a19aaf0bf1d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:42.985159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e069131-8a07-4344-88aa-6424ffa3ef93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:42.985840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e069131-8a07-4344-88aa-6424ffa3ef93 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:42.986117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e069131-8a07-4344-88aa-6424ffa3ef93 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:42.986389 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e069131-8a07-4344-88aa-6424ffa3ef93 None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01/users/e71ead94768141dc94d90c67625c3ac2/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:42.993089 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e069131-8a07-4344-88aa-6424ffa3ef93 None admin] RBAC: Authorizing `identity:create_grant(project_id=25808aef0f69482dbf49592f737cff01, user_id=e71ead94768141dc94d90c67625c3ac2, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:42.994568 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e069131-8a07-4344-88aa-6424ffa3ef93 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:43.046252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f24f34-2639-456c-8d9b-ea2b0da6a633 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:43.077495 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:43.083403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a2c69-c503-4d6c-9855-0ec6316c71c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:43.083982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a2c69-c503-4d6c-9855-0ec6316c71c4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:43.084259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a2c69-c503-4d6c-9855-0ec6316c71c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:43.084532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a2c69-c503-4d6c-9855-0ec6316c71c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:43.085194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a2c69-c503-4d6c-9855-0ec6316c71c4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:43.085496 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a2c69-c503-4d6c-9855-0ec6316c71c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:43.086937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a2c69-c503-4d6c-9855-0ec6316c71c4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:43.095812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 65/129] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:43:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:43.096620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a2c69-c503-4d6c-9855-0ec6316c71c4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:43.110193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 65/130] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:43:42 2022] PUT /identity/v3/projects/25808aef0f69482dbf49592f737cff01/users/e71ead94768141dc94d90c67625c3ac2/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:43:43.111335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e069131-8a07-4344-88aa-6424ffa3ef93 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:43.316809 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e069131-8a07-4344-88aa-6424ffa3ef93 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:43.344844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a2c69-c503-4d6c-9855-0ec6316c71c4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:43.387280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a537e41-7a86-4465-80e6-c73d799530ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:43.388329 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a537e41-7a86-4465-80e6-c73d799530ac None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:43.388812 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a537e41-7a86-4465-80e6-c73d799530ac None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:43.389572 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a537e41-7a86-4465-80e6-c73d799530ac None admin] PATH_INFO: `/v3/users/732dc764ad1742e3935aabb4dce1ae5d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:43.394451 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:43.422583 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a537e41-7a86-4465-80e6-c73d799530ac None admin] RBAC: Authorizing `identity:get_user(user_id=732dc764ad1742e3935aabb4dce1ae5d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:43.423588 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb33f18-e2d7-424e-9368-bdcf1342fa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:43.424628 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb33f18-e2d7-424e-9368-bdcf1342fa68 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:43.425133 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb33f18-e2d7-424e-9368-bdcf1342fa68 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:43.426331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb33f18-e2d7-424e-9368-bdcf1342fa68 None admin] PATH_INFO: `/v3/users/9b018c400b114999854f81cf97dcfca1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:43.428074 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a537e41-7a86-4465-80e6-c73d799530ac None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:43.429249 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb33f18-e2d7-424e-9368-bdcf1342fa68 None admin] RBAC: Authorizing `identity:get_user(user_id=9b018c400b114999854f81cf97dcfca1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:43.431333 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 66/131] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:43 2022] GET /identity/v3/users/732dc764ad1742e3935aabb4dce1ae5d => generated 289 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:43.433154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb33f18-e2d7-424e-9368-bdcf1342fa68 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:43.435335 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 66/132] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:43 2022] GET /identity/v3/users/9b018c400b114999854f81cf97dcfca1 => generated 275 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:43.440943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a537e41-7a86-4465-80e6-c73d799530ac None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:43.443572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb33f18-e2d7-424e-9368-bdcf1342fa68 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:43.528450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:43.573963 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb33f18-e2d7-424e-9368-bdcf1342fa68 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:43.616210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0191e-ea96-4ad4-86d7-8e9e276c3fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:43.616911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0191e-ea96-4ad4-86d7-8e9e276c3fdd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:43.617175 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0191e-ea96-4ad4-86d7-8e9e276c3fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:43.617452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0191e-ea96-4ad4-86d7-8e9e276c3fdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:43.697200 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf0191e-ea96-4ad4-86d7-8e9e276c3fdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:43.698797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf0191e-ea96-4ad4-86d7-8e9e276c3fdd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:43.713609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a537e41-7a86-4465-80e6-c73d799530ac None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:43.802008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:43.811717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8d542-5384-46bf-9a6f-adb96a7ed54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:43.812410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8d542-5384-46bf-9a6f-adb96a7ed54b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:43.812693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8d542-5384-46bf-9a6f-adb96a7ed54b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:43.812965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd8d542-5384-46bf-9a6f-adb96a7ed54b None admin] PATH_INFO: `/v3/projects/22c186e05ced4a909333963ed82fbddb` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:43.814104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8d542-5384-46bf-9a6f-adb96a7ed54b None admin] RBAC: Authorizing `identity:get_project(project_id=22c186e05ced4a909333963ed82fbddb)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:43.815548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd8d542-5384-46bf-9a6f-adb96a7ed54b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:43.825617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 67/133] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:43 2022] GET /identity/v3/projects/22c186e05ced4a909333963ed82fbddb => generated 312 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:43.830960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8d542-5384-46bf-9a6f-adb96a7ed54b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:43.856045 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 67/134] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:43:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:43:43.865097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0191e-ea96-4ad4-86d7-8e9e276c3fdd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:44.003134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:44.015052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0191e-ea96-4ad4-86d7-8e9e276c3fdd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:44.065514 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed6178-670c-428d-9853-9855c431adca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:44.066448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed6178-670c-428d-9853-9855c431adca None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:44.066911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed6178-670c-428d-9853-9855c431adca None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:44.067925 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed6178-670c-428d-9853-9855c431adca None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:44.071131 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed6178-670c-428d-9853-9855c431adca None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:44.073153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ed6178-670c-428d-9853-9855c431adca None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:44.077549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 68/135] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:43 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:44.079161 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763985d5-8523-4a70-9750-e570e6f93ca6 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:44.079621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763985d5-8523-4a70-9750-e570e6f93ca6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:44.080093 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763985d5-8523-4a70-9750-e570e6f93ca6 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:44.081868 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 69/136] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:44.087088 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763985d5-8523-4a70-9750-e570e6f93ca6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:44.095204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd8d542-5384-46bf-9a6f-adb96a7ed54b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:44.142726 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09388fbf-6a8a-4051-9b67-ceb7696fa14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:44.143783 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09388fbf-6a8a-4051-9b67-ceb7696fa14a None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:44.144294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09388fbf-6a8a-4051-9b67-ceb7696fa14a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:44.144768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09388fbf-6a8a-4051-9b67-ceb7696fa14a None admin] PATH_INFO: `/v3/projects/22c186e05ced4a909333963ed82fbddb/users/9b018c400b114999854f81cf97dcfca1/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:44.146757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:44.147181 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09388fbf-6a8a-4051-9b67-ceb7696fa14a None admin] RBAC: Authorizing `identity:create_grant(project_id=22c186e05ced4a909333963ed82fbddb, user_id=9b018c400b114999854f81cf97dcfca1, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:44.149616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09388fbf-6a8a-4051-9b67-ceb7696fa14a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:44.224794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763985d5-8523-4a70-9750-e570e6f93ca6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:44.241118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 68/137] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:43:43 2022] PUT /identity/v3/projects/22c186e05ced4a909333963ed82fbddb/users/9b018c400b114999854f81cf97dcfca1/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:43:44.242594 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f026d21-8451-4741-a74f-d252229c0c36 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:44.243006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f026d21-8451-4741-a74f-d252229c0c36 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:44.243513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f026d21-8451-4741-a74f-d252229c0c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:44.287052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daf818-fdc6-479a-a848-1113cf838032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:44.287758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf818-fdc6-479a-a848-1113cf838032 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:44.288126 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf818-fdc6-479a-a848-1113cf838032 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:44.288227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9f026d21-8451-4741-a74f-d252229c0c36 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:44.288398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf818-fdc6-479a-a848-1113cf838032 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:44.289280 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7daf818-fdc6-479a-a848-1113cf838032 None admin] RBAC: Adding query filter params (role.id=6a413519e66945239f17395ca12293c2, scope.project.id=25808aef0f69482dbf49592f737cff01, user.id=732dc764ad1742e3935aabb4dce1ae5d) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:44.289381 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9f026d21-8451-4741-a74f-d252229c0c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:44.289590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7daf818-fdc6-479a-a848-1113cf838032 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:44.291241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7daf818-fdc6-479a-a848-1113cf838032 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:44.301599 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 70/138] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:43:44 2022] GET /identity/v3/role_assignments?user.id=732dc764ad1742e3935aabb4dce1ae5d&scope.project.id=25808aef0f69482dbf49592f737cff01&role.id=6a413519e66945239f17395ca12293c2 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:44.403997 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 69/139] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:44.973031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daf818-fdc6-479a-a848-1113cf838032 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:45.088568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:45.096717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daf818-fdc6-479a-a848-1113cf838032 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:45.141271 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a675240-f1e6-4522-aec0-7c2e8fe7edce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:45.141920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a675240-f1e6-4522-aec0-7c2e8fe7edce None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:45.142202 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a675240-f1e6-4522-aec0-7c2e8fe7edce None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:45.142482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a675240-f1e6-4522-aec0-7c2e8fe7edce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:45.156838 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a675240-f1e6-4522-aec0-7c2e8fe7edce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4a675240-f1e6-4522-aec0-7c2e8fe7edce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:45.163353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:45.164194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:45.165041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:45.165041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:45.165041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:45.165041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:45.165041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:45.165041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:45.165041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:45.165041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:45.165041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:45.165041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:45.165041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:43:45.165041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:43:45.165041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 71/140] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:45.172546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f026d21-8451-4741-a74f-d252229c0c36 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:45.263424 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:45.334990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f026d21-8451-4741-a74f-d252229c0c36 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:45.383595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff40502-58f4-48b4-900a-ccc8e5d4a85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:45.384315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff40502-58f4-48b4-900a-ccc8e5d4a85a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:45.384601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff40502-58f4-48b4-900a-ccc8e5d4a85a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:45.384872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff40502-58f4-48b4-900a-ccc8e5d4a85a None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:45.385537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff40502-58f4-48b4-900a-ccc8e5d4a85a None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:45.385843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff40502-58f4-48b4-900a-ccc8e5d4a85a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:45.387235 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff40502-58f4-48b4-900a-ccc8e5d4a85a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:45.399395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 70/141] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:43:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:45.405136 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a675240-f1e6-4522-aec0-7c2e8fe7edce None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:45.478738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6c56b-a8bd-40b9-9be8-53b2f066d04f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:45.479057 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6c56b-a8bd-40b9-9be8-53b2f066d04f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:45.479302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa6c56b-a8bd-40b9-9be8-53b2f066d04f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:45.480961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 71/142] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:45.487261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279dc1f-7c73-4295-9753-0e4f06a50aa3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:45.487547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279dc1f-7c73-4295-9753-0e4f06a50aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:45.487845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279dc1f-7c73-4295-9753-0e4f06a50aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:45.515836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4279dc1f-7c73-4295-9753-0e4f06a50aa3 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:45.516974 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4279dc1f-7c73-4295-9753-0e4f06a50aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:45.588509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a675240-f1e6-4522-aec0-7c2e8fe7edce None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:45.655774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 72/143] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:45.657975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08dd08-345e-4a40-802b-5f6be336fbf8 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:45.658263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08dd08-345e-4a40-802b-5f6be336fbf8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:45.658539 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a08dd08-345e-4a40-802b-5f6be336fbf8 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:45.659806 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 73/144] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:45.663378 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcf428-dd3c-46bc-b0c3-9d00b90e3af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:45.664130 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcf428-dd3c-46bc-b0c3-9d00b90e3af4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:45.664408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcf428-dd3c-46bc-b0c3-9d00b90e3af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:45.664678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcf428-dd3c-46bc-b0c3-9d00b90e3af4 None admin] PATH_INFO: `/v3/users/9b018c400b114999854f81cf97dcfca1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:45.666296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f7bde-d323-448a-9152-a2fb94b6d672 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:45.666653 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f7bde-d323-448a-9152-a2fb94b6d672 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:45.666914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759f7bde-d323-448a-9152-a2fb94b6d672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:45.668485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcf428-dd3c-46bc-b0c3-9d00b90e3af4 None admin] RBAC: Authorizing `identity:get_user(user_id=9b018c400b114999854f81cf97dcfca1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:45.670951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcf428-dd3c-46bc-b0c3-9d00b90e3af4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:45.672688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 72/145] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:45 2022] GET /identity/v3/users/9b018c400b114999854f81cf97dcfca1 => generated 275 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:45.680410 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcf428-dd3c-46bc-b0c3-9d00b90e3af4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:45.708488 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-759f7bde-d323-448a-9152-a2fb94b6d672 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:45.709563 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-759f7bde-d323-448a-9152-a2fb94b6d672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:45.744055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:45.796117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:45.855677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 74/146] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:45.883420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcf428-dd3c-46bc-b0c3-9d00b90e3af4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:45.937450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2a210-bfcb-4a48-8d5e-a89c8c47a1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:45.938292 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2a210-bfcb-4a48-8d5e-a89c8c47a1cc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:45.938712 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2a210-bfcb-4a48-8d5e-a89c8c47a1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:45.939119 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe2a210-bfcb-4a48-8d5e-a89c8c47a1cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:46.047241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2a210-bfcb-4a48-8d5e-a89c8c47a1cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:46.048671 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe2a210-bfcb-4a48-8d5e-a89c8c47a1cc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:46.081986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:46.174876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 73/147] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:43:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:43:46.181133 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f7bde-d323-448a-9152-a2fb94b6d672 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:46.207242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2a210-bfcb-4a48-8d5e-a89c8c47a1cc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:46.344253 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759f7bde-d323-448a-9152-a2fb94b6d672 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:46.373407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe2a210-bfcb-4a48-8d5e-a89c8c47a1cc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:46.402810 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:46.417760 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00410b6b-1b5a-432d-97fa-978101a678b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:46.418471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00410b6b-1b5a-432d-97fa-978101a678b2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:46.419434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00410b6b-1b5a-432d-97fa-978101a678b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:46.419434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00410b6b-1b5a-432d-97fa-978101a678b2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:46.420419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00410b6b-1b5a-432d-97fa-978101a678b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:46.422428 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00410b6b-1b5a-432d-97fa-978101a678b2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:46.424048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 74/148] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:43:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:46.424811 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00410b6b-1b5a-432d-97fa-978101a678b2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:46.425786 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efffbe-a0a6-45e6-9786-9e10e088c608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:46.426469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efffbe-a0a6-45e6-9786-9e10e088c608 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:46.426750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efffbe-a0a6-45e6-9786-9e10e088c608 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:46.427020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1efffbe-a0a6-45e6-9786-9e10e088c608 None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:46.428202 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efffbe-a0a6-45e6-9786-9e10e088c608 None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:46.430053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1efffbe-a0a6-45e6-9786-9e10e088c608 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:46.431782 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 75/149] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:46 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:46.432518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efffbe-a0a6-45e6-9786-9e10e088c608 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:46.534163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:46.588318 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00410b6b-1b5a-432d-97fa-978101a678b2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:46.643941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1efffbe-a0a6-45e6-9786-9e10e088c608 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:46.644265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648191bc-9ac4-49d7-9032-ee5ff15b040e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:46.644850 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648191bc-9ac4-49d7-9032-ee5ff15b040e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:46.645274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648191bc-9ac4-49d7-9032-ee5ff15b040e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:46.645776 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648191bc-9ac4-49d7-9032-ee5ff15b040e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:46.659145 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648191bc-9ac4-49d7-9032-ee5ff15b040e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-648191bc-9ac4-49d7-9032-ee5ff15b040e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:46.666571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:46.667399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:46.668167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:46.668167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:46.668167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:46.668167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:46.668167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:46.668167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:46.668167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:46.668167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:46.668167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:46.668167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:46.668167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:43:46.668167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:43:46.671858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 75/150] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:43:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:46.672709 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648191bc-9ac4-49d7-9032-ee5ff15b040e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:46.689597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2015bc71-5df7-4742-b071-bdd5cf548fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:46.690310 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015bc71-5df7-4742-b071-bdd5cf548fd1 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:46.690596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015bc71-5df7-4742-b071-bdd5cf548fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:46.690872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2015bc71-5df7-4742-b071-bdd5cf548fd1 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:46.691531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015bc71-5df7-4742-b071-bdd5cf548fd1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:46.694168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2015bc71-5df7-4742-b071-bdd5cf548fd1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:46.735445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:46.787154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648191bc-9ac4-49d7-9032-ee5ff15b040e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:46.811889 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 76/151] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:43:46 2022] POST /identity/v3/users => generated 283 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:43:46.813273 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88be1e-892e-4be0-b565-1886335cb3a3 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:46.813567 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88be1e-892e-4be0-b565-1886335cb3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:46.813835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88be1e-892e-4be0-b565-1886335cb3a3 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:46.815116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 77/152] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:46.815877 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88be1e-892e-4be0-b565-1886335cb3a3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:46.840150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe3514d-cf55-4aec-8410-0babf026ec69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:46.840848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3514d-cf55-4aec-8410-0babf026ec69 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:46.841134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3514d-cf55-4aec-8410-0babf026ec69 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:46.841413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe3514d-cf55-4aec-8410-0babf026ec69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:46.842174 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3514d-cf55-4aec-8410-0babf026ec69 None admin] RBAC: Adding query filter params (role.id=1c983c7e32e44c61be3996c28399e617, scope.project.id=25808aef0f69482dbf49592f737cff01, user.id=9b018c400b114999854f81cf97dcfca1) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:46.842489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3514d-cf55-4aec-8410-0babf026ec69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:46.844097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe3514d-cf55-4aec-8410-0babf026ec69 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:46.853353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 76/153] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:43:46 2022] GET /identity/v3/role_assignments?user.id=9b018c400b114999854f81cf97dcfca1&scope.project.id=25808aef0f69482dbf49592f737cff01&role.id=1c983c7e32e44c61be3996c28399e617 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:46.854931 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11716c95-d30f-4ef6-a797-7a03537c7756 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:46.855218 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11716c95-d30f-4ef6-a797-7a03537c7756 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:46.855807 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11716c95-d30f-4ef6-a797-7a03537c7756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:46.880311 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-11716c95-d30f-4ef6-a797-7a03537c7756 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:46.881800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-11716c95-d30f-4ef6-a797-7a03537c7756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:47.002934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 77/154] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:47.041278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:47.050880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88be1e-892e-4be0-b565-1886335cb3a3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:47.105051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31150452-b311-4f95-aa9e-f34e7d5e5835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:47.105677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31150452-b311-4f95-aa9e-f34e7d5e5835 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:47.105962 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31150452-b311-4f95-aa9e-f34e7d5e5835 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:47.106236 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31150452-b311-4f95-aa9e-f34e7d5e5835 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:47.106911 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31150452-b311-4f95-aa9e-f34e7d5e5835 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:47.107246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31150452-b311-4f95-aa9e-f34e7d5e5835 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:47.110097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31150452-b311-4f95-aa9e-f34e7d5e5835 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:47.123544 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 78/155] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:43:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:47.129760 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11716c95-d30f-4ef6-a797-7a03537c7756 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:47.210822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:47.284590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11716c95-d30f-4ef6-a797-7a03537c7756 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:47.313909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35ee61-6030-43e8-82df-6dd7cecb9c7d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:47.314395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35ee61-6030-43e8-82df-6dd7cecb9c7d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:47.314465 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35ee61-6030-43e8-82df-6dd7cecb9c7d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:47.318445 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 79/156] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:43:47.323970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff52d9c-4730-464c-93be-3b76a23c1a66 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:47.324322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff52d9c-4730-464c-93be-3b76a23c1a66 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:47.324520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff52d9c-4730-464c-93be-3b76a23c1a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:47.328772 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102b1b9-1a55-41c2-9dc7-16fae2cc189d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:47.329422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102b1b9-1a55-41c2-9dc7-16fae2cc189d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:47.329695 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102b1b9-1a55-41c2-9dc7-16fae2cc189d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:47.329969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8102b1b9-1a55-41c2-9dc7-16fae2cc189d None admin] PATH_INFO: `/v3/users/9b018c400b114999854f81cf97dcfca1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:47.331107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102b1b9-1a55-41c2-9dc7-16fae2cc189d None admin] RBAC: Authorizing `identity:get_user(user_id=9b018c400b114999854f81cf97dcfca1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:47.333217 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8102b1b9-1a55-41c2-9dc7-16fae2cc189d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:47.335322 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 78/157] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:47 2022] GET /identity/v3/users/9b018c400b114999854f81cf97dcfca1 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:47.341513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102b1b9-1a55-41c2-9dc7-16fae2cc189d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:47.360595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cff52d9c-4730-464c-93be-3b76a23c1a66 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:47.362139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cff52d9c-4730-464c-93be-3b76a23c1a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:47.462293 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 80/158] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:47.463252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff52d9c-4730-464c-93be-3b76a23c1a66 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:47.470096 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8102b1b9-1a55-41c2-9dc7-16fae2cc189d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:47.487290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:47.551044 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c869801-105e-4576-8d55-041f5f739916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:47.552816 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c869801-105e-4576-8d55-041f5f739916 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:47.553162 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c869801-105e-4576-8d55-041f5f739916 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:47.553435 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c869801-105e-4576-8d55-041f5f739916 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:47.597838 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:47.613049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff52d9c-4730-464c-93be-3b76a23c1a66 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:47.653417 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd639c6-9bbc-4d8b-8510-03764f2567d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:47.654067 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd639c6-9bbc-4d8b-8510-03764f2567d0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:47.654367 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd639c6-9bbc-4d8b-8510-03764f2567d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:47.654671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd639c6-9bbc-4d8b-8510-03764f2567d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:47.662528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c869801-105e-4576-8d55-041f5f739916 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:47.665181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c869801-105e-4576-8d55-041f5f739916 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:47.670595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd639c6-9bbc-4d8b-8510-03764f2567d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-fcd639c6-9bbc-4d8b-8510-03764f2567d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:47.677078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:47.678925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:47.680848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:47.680848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:47.680848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:47.680848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:47.680848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:47.680848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:47.680848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:47.680848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:47.680848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:47.680848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:47.680848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:47.680848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:43:47.685213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 81/159] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:43:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:47.687051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad3e3f-6742-4c9e-a29e-e4d78fa3bbe3 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:47.687934 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad3e3f-6742-4c9e-a29e-e4d78fa3bbe3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:47.688426 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad3e3f-6742-4c9e-a29e-e4d78fa3bbe3 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:47.691247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 82/160] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:47.695011 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad3e3f-6742-4c9e-a29e-e4d78fa3bbe3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:47.805916 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 79/161] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:43:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:43:47.807184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee8a87-efbf-44ec-bf28-01158248f437 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:47.807462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee8a87-efbf-44ec-bf28-01158248f437 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:47.811872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee8a87-efbf-44ec-bf28-01158248f437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:47.838743 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-09ee8a87-efbf-44ec-bf28-01158248f437 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:47.840005 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-09ee8a87-efbf-44ec-bf28-01158248f437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:47.848422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:47.849039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad3e3f-6742-4c9e-a29e-e4d78fa3bbe3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:47.893467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d23d8f-9f61-47a5-9b39-230b5e4cecf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:47.894192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d23d8f-9f61-47a5-9b39-230b5e4cecf2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:47.894679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d23d8f-9f61-47a5-9b39-230b5e4cecf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:47.895724 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d23d8f-9f61-47a5-9b39-230b5e4cecf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:47.896447 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d23d8f-9f61-47a5-9b39-230b5e4cecf2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:47.896973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d23d8f-9f61-47a5-9b39-230b5e4cecf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:47.898741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d23d8f-9f61-47a5-9b39-230b5e4cecf2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:47.910585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 83/162] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:47.912205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d23d8f-9f61-47a5-9b39-230b5e4cecf2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:47.932927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 80/163] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:47.933428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:47.933875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee8a87-efbf-44ec-bf28-01158248f437 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:48.109961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d23d8f-9f61-47a5-9b39-230b5e4cecf2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:48.110132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee8a87-efbf-44ec-bf28-01158248f437 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:48.171275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b9875-186c-4d31-80b4-f47aca3eeea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:48.172065 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b9875-186c-4d31-80b4-f47aca3eeea3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:48.172351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b9875-186c-4d31-80b4-f47aca3eeea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:48.172626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46b9875-186c-4d31-80b4-f47aca3eeea3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:48.182478 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce94bf-93f0-4237-835d-2eaa85faba28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:48.183132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce94bf-93f0-4237-835d-2eaa85faba28 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:48.183415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce94bf-93f0-4237-835d-2eaa85faba28 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:48.183707 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce94bf-93f0-4237-835d-2eaa85faba28 None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:48.184769 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46b9875-186c-4d31-80b4-f47aca3eeea3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:48.188357 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce94bf-93f0-4237-835d-2eaa85faba28 None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e46b9875-186c-4d31-80b4-f47aca3eeea3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:48.189653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:48.190475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:48.191301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:48.191301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:48.191301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:48.191301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:48.191301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:48.191301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:48.191301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:48.191301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:48.191301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:48.191301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:48.191301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:43:48.191301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:43:48.192702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce94bf-93f0-4237-835d-2eaa85faba28 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:48.192783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 81/164] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:48.192849 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 84/165] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:47 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:48.192910 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b9875-186c-4d31-80b4-f47aca3eeea3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:48.200594 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce94bf-93f0-4237-835d-2eaa85faba28 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:48.203788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:48.358366 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:48.383453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce94bf-93f0-4237-835d-2eaa85faba28 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:48.433731 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9c6ff-4fac-4cda-885e-7a4512ffe28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:48.434711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9c6ff-4fac-4cda-885e-7a4512ffe28f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:48.435133 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9c6ff-4fac-4cda-885e-7a4512ffe28f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:48.435543 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9c6ff-4fac-4cda-885e-7a4512ffe28f None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:48.436394 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9c6ff-4fac-4cda-885e-7a4512ffe28f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:48.436858 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9c6ff-4fac-4cda-885e-7a4512ffe28f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:48.438483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9c6ff-4fac-4cda-885e-7a4512ffe28f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:48.447670 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46b9875-186c-4d31-80b4-f47aca3eeea3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:48.456164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 85/166] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:43:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:48.457270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9c6ff-4fac-4cda-885e-7a4512ffe28f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:48.484112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea6bca-08ee-4d19-9754-2870fd74b409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:48.484112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea6bca-08ee-4d19-9754-2870fd74b409 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:48.484112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea6bca-08ee-4d19-9754-2870fd74b409 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:48.484112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ea6bca-08ee-4d19-9754-2870fd74b409 None admin] PATH_INFO: `/v3/users/e71ead94768141dc94d90c67625c3ac2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:48.484502 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea6bca-08ee-4d19-9754-2870fd74b409 None admin] RBAC: Authorizing `identity:get_user(user_id=e71ead94768141dc94d90c67625c3ac2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:48.484502 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ea6bca-08ee-4d19-9754-2870fd74b409 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:48.485791 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 82/167] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:48 2022] GET /identity/v3/users/e71ead94768141dc94d90c67625c3ac2 => generated 247 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:48.486660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea6bca-08ee-4d19-9754-2870fd74b409 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:48.608051 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:48.698537 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ea6bca-08ee-4d19-9754-2870fd74b409 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:48.698783 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9c6ff-4fac-4cda-885e-7a4512ffe28f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:48.716052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bf1d0-b108-4b84-82d8-3722595b9070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:48.716695 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bf1d0-b108-4b84-82d8-3722595b9070 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:48.716974 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bf1d0-b108-4b84-82d8-3722595b9070 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:48.717248 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17bf1d0-b108-4b84-82d8-3722595b9070 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:48.738042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17bf1d0-b108-4b84-82d8-3722595b9070 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f17bf1d0-b108-4b84-82d8-3722595b9070 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:48.744952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:48.745853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:48.746635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:48.746635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:48.746635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:48.746635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:48.746635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:48.746635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:48.746635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:48.746635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:48.746635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:48.746635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:48.746635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:43:48.746635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:43:48.751717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 83/168] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:48.752584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bf1d0-b108-4b84-82d8-3722595b9070 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:48.757595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6b1d1-b846-4713-ba7f-752b48a4dcb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:48.758152 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6b1d1-b846-4713-ba7f-752b48a4dcb6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:48.758434 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6b1d1-b846-4713-ba7f-752b48a4dcb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:48.758709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6b1d1-b846-4713-ba7f-752b48a4dcb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:48.759494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6b1d1-b846-4713-ba7f-752b48a4dcb6 None admin] RBAC: Adding query filter params (role.id=30bb990dfe5c45fbbfed21c08c7859b4, scope.project.id=25808aef0f69482dbf49592f737cff01, user.id=9b018c400b114999854f81cf97dcfca1) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:48.759847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6b1d1-b846-4713-ba7f-752b48a4dcb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:48.761278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6b1d1-b846-4713-ba7f-752b48a4dcb6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:48.776818 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 86/169] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:43:48 2022] GET /identity/v3/role_assignments?user.id=9b018c400b114999854f81cf97dcfca1&scope.project.id=25808aef0f69482dbf49592f737cff01&role.id=30bb990dfe5c45fbbfed21c08c7859b4 => generated 608 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:48.777644 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6b1d1-b846-4713-ba7f-752b48a4dcb6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:48.788660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:48.920224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17bf1d0-b108-4b84-82d8-3722595b9070 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:48.951023 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6b1d1-b846-4713-ba7f-752b48a4dcb6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:48.992382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b266b97-e279-4979-838b-0afa20bdd8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:48.993005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b266b97-e279-4979-838b-0afa20bdd8ea None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:48.993287 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b266b97-e279-4979-838b-0afa20bdd8ea None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:48.993557 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b266b97-e279-4979-838b-0afa20bdd8ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:49.022252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:49.040937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61c4a5-9786-4b66-a897-e9e7a57e48ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:49.041592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61c4a5-9786-4b66-a897-e9e7a57e48ac None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:49.041869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61c4a5-9786-4b66-a897-e9e7a57e48ac None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:49.042139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f61c4a5-9786-4b66-a897-e9e7a57e48ac None admin] PATH_INFO: `/v3/users/732dc764ad1742e3935aabb4dce1ae5d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:49.046305 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61c4a5-9786-4b66-a897-e9e7a57e48ac None admin] RBAC: Authorizing `identity:get_user(user_id=732dc764ad1742e3935aabb4dce1ae5d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:49.047911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f61c4a5-9786-4b66-a897-e9e7a57e48ac None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:49.049885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 84/170] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:48 2022] GET /identity/v3/users/732dc764ad1742e3935aabb4dce1ae5d => generated 289 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:49.050703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61c4a5-9786-4b66-a897-e9e7a57e48ac None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:49.075127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b266b97-e279-4979-838b-0afa20bdd8ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:49.076673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b266b97-e279-4979-838b-0afa20bdd8ea None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:49.080023 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:49.194334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 87/171] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:43:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:43:49.194538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f61c4a5-9786-4b66-a897-e9e7a57e48ac None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:49.195181 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b266b97-e279-4979-838b-0afa20bdd8ea None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:49.236375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99070ebc-58f4-4f89-98f8-b84b7cf50182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:49.236946 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99070ebc-58f4-4f89-98f8-b84b7cf50182 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:49.237226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99070ebc-58f4-4f89-98f8-b84b7cf50182 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:49.237496 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99070ebc-58f4-4f89-98f8-b84b7cf50182 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:49.238164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99070ebc-58f4-4f89-98f8-b84b7cf50182 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:49.238467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99070ebc-58f4-4f89-98f8-b84b7cf50182 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:49.240926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99070ebc-58f4-4f89-98f8-b84b7cf50182 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:49.249662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 85/172] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:43:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:49.250493 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99070ebc-58f4-4f89-98f8-b84b7cf50182 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:49.358306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b266b97-e279-4979-838b-0afa20bdd8ea None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:49.364536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:49.370843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:49.426753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99070ebc-58f4-4f89-98f8-b84b7cf50182 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:49.429097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb5c0e-6589-443a-b9ff-ee19b507c47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:49.429737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb5c0e-6589-443a-b9ff-ee19b507c47a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:49.430054 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb5c0e-6589-443a-b9ff-ee19b507c47a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:49.430287 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefb5c0e-6589-443a-b9ff-ee19b507c47a None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:49.431521 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb5c0e-6589-443a-b9ff-ee19b507c47a None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:49.432996 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefb5c0e-6589-443a-b9ff-ee19b507c47a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:49.434800 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 88/173] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:49 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:49.435381 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb5c0e-6589-443a-b9ff-ee19b507c47a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:49.500327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23bafe-98a9-43c6-af4e-c4409a7a10f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:49.500896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23bafe-98a9-43c6-af4e-c4409a7a10f1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:49.501175 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23bafe-98a9-43c6-af4e-c4409a7a10f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:49.501447 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af23bafe-98a9-43c6-af4e-c4409a7a10f1 None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:49.502855 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23bafe-98a9-43c6-af4e-c4409a7a10f1 None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:49.504880 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af23bafe-98a9-43c6-af4e-c4409a7a10f1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:49.508600 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 86/174] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:49 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:49.509439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23bafe-98a9-43c6-af4e-c4409a7a10f1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:49.596927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefb5c0e-6589-443a-b9ff-ee19b507c47a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:49.661329 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56ce34-d193-4d04-bf9a-cb29f49da95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:49.662037 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56ce34-d193-4d04-bf9a-cb29f49da95a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:49.662316 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56ce34-d193-4d04-bf9a-cb29f49da95a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:49.662581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56ce34-d193-4d04-bf9a-cb29f49da95a None admin] PATH_INFO: `/v3/users/9b018c400b114999854f81cf97dcfca1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:49.666729 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56ce34-d193-4d04-bf9a-cb29f49da95a None admin] RBAC: Authorizing `identity:get_user(user_id=9b018c400b114999854f81cf97dcfca1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:49.668196 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56ce34-d193-4d04-bf9a-cb29f49da95a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:49.678213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 89/175] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:49 2022] GET /identity/v3/users/9b018c400b114999854f81cf97dcfca1 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:49.679054 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56ce34-d193-4d04-bf9a-cb29f49da95a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:49.726603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af23bafe-98a9-43c6-af4e-c4409a7a10f1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:49.760327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:49.784029 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:49.803985 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31b7b0-ae8e-4c1e-9727-bda7f1a0d0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:49.823677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31b7b0-ae8e-4c1e-9727-bda7f1a0d0a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:49.823677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31b7b0-ae8e-4c1e-9727-bda7f1a0d0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:49.823677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31b7b0-ae8e-4c1e-9727-bda7f1a0d0a6 None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01/users/732dc764ad1742e3935aabb4dce1ae5d/roles/6a413519e66945239f17395ca12293c2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:49.823677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31b7b0-ae8e-4c1e-9727-bda7f1a0d0a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=25808aef0f69482dbf49592f737cff01, user_id=732dc764ad1742e3935aabb4dce1ae5d, role_id=6a413519e66945239f17395ca12293c2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:49.823677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31b7b0-ae8e-4c1e-9727-bda7f1a0d0a6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:49.857174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56ce34-d193-4d04-bf9a-cb29f49da95a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:49.893760 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b6340-27cb-4e90-afc1-9a0adefb3d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:49.894429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b6340-27cb-4e90-afc1-9a0adefb3d6a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:49.894716 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b6340-27cb-4e90-afc1-9a0adefb3d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:49.895610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713b6340-27cb-4e90-afc1-9a0adefb3d6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:49.896713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b6340-27cb-4e90-afc1-9a0adefb3d6a None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=25808aef0f69482dbf49592f737cff01, user.id=e71ead94768141dc94d90c67625c3ac2) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:49.897031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b6340-27cb-4e90-afc1-9a0adefb3d6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:49.898477 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713b6340-27cb-4e90-afc1-9a0adefb3d6a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:49.904921 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 90/176] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:43:49 2022] GET /identity/v3/role_assignments?user.id=e71ead94768141dc94d90c67625c3ac2&scope.project.id=25808aef0f69482dbf49592f737cff01&role.id=07626899472b4e32adc0c71ce796a25d => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:49.905726 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b6340-27cb-4e90-afc1-9a0adefb3d6a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:49.944968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 87/177] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:43:49 2022] PUT /identity/v3/projects/25808aef0f69482dbf49592f737cff01/users/732dc764ad1742e3935aabb4dce1ae5d/roles/6a413519e66945239f17395ca12293c2 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:43:50.030126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713b6340-27cb-4e90-afc1-9a0adefb3d6a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:50.032712 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:50.070545 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd743d-f21a-4f2e-a864-2acbc7f3e796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:50.071197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd743d-f21a-4f2e-a864-2acbc7f3e796 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:50.071479 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd743d-f21a-4f2e-a864-2acbc7f3e796 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:50.071790 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd743d-f21a-4f2e-a864-2acbc7f3e796 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:50.154488 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd743d-f21a-4f2e-a864-2acbc7f3e796 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:50.156098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd743d-f21a-4f2e-a864-2acbc7f3e796 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:50.278855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 91/178] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:43:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:43:50.284691 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31b7b0-ae8e-4c1e-9727-bda7f1a0d0a6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:50.371793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bb02a-e08e-4217-83fe-b96734c76c34 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:50.372319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bb02a-e08e-4217-83fe-b96734c76c34 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:50.372319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1bb02a-e08e-4217-83fe-b96734c76c34 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:50.373818 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 92/179] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:50.379053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5396a7-faaf-46aa-bb2f-19bb51818d9e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:50.380250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5396a7-faaf-46aa-bb2f-19bb51818d9e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:50.380637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5396a7-faaf-46aa-bb2f-19bb51818d9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:50.388312 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:50.403629 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1c5396a7-faaf-46aa-bb2f-19bb51818d9e None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:50.408442 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31b7b0-ae8e-4c1e-9727-bda7f1a0d0a6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:50.408555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1c5396a7-faaf-46aa-bb2f-19bb51818d9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:50.446755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ad2df-7a3e-48e6-9326-879ad6dbc629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:50.447613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ad2df-7a3e-48e6-9326-879ad6dbc629 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:50.447926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ad2df-7a3e-48e6-9326-879ad6dbc629 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:50.448581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9ad2df-7a3e-48e6-9326-879ad6dbc629 None admin] PATH_INFO: `/v3/projects/22c186e05ced4a909333963ed82fbddb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:50.450086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ad2df-7a3e-48e6-9326-879ad6dbc629 None admin] RBAC: Authorizing `identity:get_project(project_id=22c186e05ced4a909333963ed82fbddb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:50.451799 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9ad2df-7a3e-48e6-9326-879ad6dbc629 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:50.454067 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 88/180] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:50 2022] GET /identity/v3/projects/22c186e05ced4a909333963ed82fbddb => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:50.461637 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ad2df-7a3e-48e6-9326-879ad6dbc629 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:50.501525 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 93/181] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:43:50.527750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:50.575214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9ad2df-7a3e-48e6-9326-879ad6dbc629 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:50.616056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d0826e-33da-4b1a-a2b5-0bef69d46ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:50.616687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0826e-33da-4b1a-a2b5-0bef69d46ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:50.617040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0826e-33da-4b1a-a2b5-0bef69d46ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:50.617333 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d0826e-33da-4b1a-a2b5-0bef69d46ea0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:50.618080 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d0826e-33da-4b1a-a2b5-0bef69d46ea0 None admin] RBAC: Adding query filter params (role.id=1c983c7e32e44c61be3996c28399e617, scope.project.id=22c186e05ced4a909333963ed82fbddb, user.id=9b018c400b114999854f81cf97dcfca1) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:50.618393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d0826e-33da-4b1a-a2b5-0bef69d46ea0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:50.619887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d0826e-33da-4b1a-a2b5-0bef69d46ea0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:50.626866 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 89/182] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:43:50 2022] GET /identity/v3/role_assignments?user.id=9b018c400b114999854f81cf97dcfca1&scope.project.id=22c186e05ced4a909333963ed82fbddb&role.id=1c983c7e32e44c61be3996c28399e617 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:50.777813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f43227-346f-47b4-a7d4-afc8ef83ff93 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:50.778130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f43227-346f-47b4-a7d4-afc8ef83ff93 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:50.778372 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f43227-346f-47b4-a7d4-afc8ef83ff93 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:50.779875 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 94/183] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:50.786653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a14c1-87ba-49c2-aa6d-6eeb582bc5de None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:50.787298 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a14c1-87ba-49c2-aa6d-6eeb582bc5de None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:50.787825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a14c1-87ba-49c2-aa6d-6eeb582bc5de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:50.789757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0999067-6b7f-42c9-bd5e-e4eb370cf016 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:50.790247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0999067-6b7f-42c9-bd5e-e4eb370cf016 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:50.790718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0999067-6b7f-42c9-bd5e-e4eb370cf016 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:50.795358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 95/184] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:50.798493 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c1f56-1436-4142-8a50-c6f0f732b6fa None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:50.799049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c1f56-1436-4142-8a50-c6f0f732b6fa None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:50.799533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c1f56-1436-4142-8a50-c6f0f732b6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:50.838308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e24a14c1-87ba-49c2-aa6d-6eeb582bc5de None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:50.840070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-240c1f56-1436-4142-8a50-c6f0f732b6fa None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:50.841530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-240c1f56-1436-4142-8a50-c6f0f732b6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:50.842658 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e24a14c1-87ba-49c2-aa6d-6eeb582bc5de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:50.944656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 90/185] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:50.945841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a14c1-87ba-49c2-aa6d-6eeb582bc5de None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:50.966617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 96/186] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:51.020207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:51.079534 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a14c1-87ba-49c2-aa6d-6eeb582bc5de None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:51.123337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6773d-779e-47f9-9ee1-e15568cabaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:51.124099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6773d-779e-47f9-9ee1-e15568cabaa0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:51.124384 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6773d-779e-47f9-9ee1-e15568cabaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:51.124655 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d6773d-779e-47f9-9ee1-e15568cabaa0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:51.136073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d6773d-779e-47f9-9ee1-e15568cabaa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f1d6773d-779e-47f9-9ee1-e15568cabaa0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:51.141289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:51.142139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:51.142914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:51.142914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:51.142914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:51.142914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:51.142914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:51.142914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:51.142914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:51.142914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:51.142914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:51.142914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:51.142914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:51.142914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:43:51.142914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 91/187] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:43:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:51.149258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c1f56-1436-4142-8a50-c6f0f732b6fa None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:51.215281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:51.254964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c1f56-1436-4142-8a50-c6f0f732b6fa None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:51.255710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6773d-779e-47f9-9ee1-e15568cabaa0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:51.293919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa53a80-8757-4dec-80a1-aebc1fa10b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:51.294788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa53a80-8757-4dec-80a1-aebc1fa10b6b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:51.295070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa53a80-8757-4dec-80a1-aebc1fa10b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:51.295345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa53a80-8757-4dec-80a1-aebc1fa10b6b None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:51.296054 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa53a80-8757-4dec-80a1-aebc1fa10b6b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:51.296360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa53a80-8757-4dec-80a1-aebc1fa10b6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:51.297795 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa53a80-8757-4dec-80a1-aebc1fa10b6b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:51.307710 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 97/188] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:51.307710 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa53a80-8757-4dec-80a1-aebc1fa10b6b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:51.385140 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d6773d-779e-47f9-9ee1-e15568cabaa0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:51.396657 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:51.420133 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01e0c6-42dc-4c78-acb1-200d907ac0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:51.421067 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01e0c6-42dc-4c78-acb1-200d907ac0ad None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:51.421353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01e0c6-42dc-4c78-acb1-200d907ac0ad None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:51.421629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01e0c6-42dc-4c78-acb1-200d907ac0ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:51.422824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01e0c6-42dc-4c78-acb1-200d907ac0ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:51.424679 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01e0c6-42dc-4c78-acb1-200d907ac0ad None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:51.427963 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 92/189] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:43:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:51.429179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01e0c6-42dc-4c78-acb1-200d907ac0ad None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:51.436899 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa53a80-8757-4dec-80a1-aebc1fa10b6b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:51.468987 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:51.477822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da030def-e581-4e73-b090-3d7b0edefbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:51.482483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da030def-e581-4e73-b090-3d7b0edefbbf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:51.482483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da030def-e581-4e73-b090-3d7b0edefbbf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:51.482483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da030def-e581-4e73-b090-3d7b0edefbbf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:51.490813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da030def-e581-4e73-b090-3d7b0edefbbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-da030def-e581-4e73-b090-3d7b0edefbbf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:51.497014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:51.497857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:51.498635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:51.498635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:51.498635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:51.498635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:51.498635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:51.498635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:51.498635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:51.498635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:51.498635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:51.498635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:51.498635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:43:51.498635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:43:51.498635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 98/190] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:43:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:51.499289 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da030def-e581-4e73-b090-3d7b0edefbbf None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:51.552684 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01e0c6-42dc-4c78-acb1-200d907ac0ad None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:51.592471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9c3c1-e8b6-4d9d-957e-7cc9ed8445fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:51.593210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9c3c1-e8b6-4d9d-957e-7cc9ed8445fd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:51.593492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9c3c1-e8b6-4d9d-957e-7cc9ed8445fd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:51.593768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9c3c1-e8b6-4d9d-957e-7cc9ed8445fd None admin] PATH_INFO: `/v3/users/136255de56114dec8740f51546436962` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:51.607978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da030def-e581-4e73-b090-3d7b0edefbbf None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:51.608881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9c3c1-e8b6-4d9d-957e-7cc9ed8445fd None admin] RBAC: Authorizing `identity:get_user(user_id=136255de56114dec8740f51546436962)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:51.610604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9c3c1-e8b6-4d9d-957e-7cc9ed8445fd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:51.617747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 93/191] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:51 2022] GET /identity/v3/users/136255de56114dec8740f51546436962 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:51.618486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9c3c1-e8b6-4d9d-957e-7cc9ed8445fd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:51.648070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6b201-cd4a-4ea5-9a4e-fc48bb5cedd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:51.648603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6b201-cd4a-4ea5-9a4e-fc48bb5cedd8 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:51.648891 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6b201-cd4a-4ea5-9a4e-fc48bb5cedd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:51.649232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6b201-cd4a-4ea5-9a4e-fc48bb5cedd8 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:51.649894 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6b201-cd4a-4ea5-9a4e-fc48bb5cedd8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:51.651335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6b201-cd4a-4ea5-9a4e-fc48bb5cedd8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:51.655786 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:51.720833 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9c3c1-e8b6-4d9d-957e-7cc9ed8445fd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:51.727640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 99/192] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:43:51 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:43:51.728324 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6b201-cd4a-4ea5-9a4e-fc48bb5cedd8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:51.776225 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342bd19-ce5f-40d4-9f54-af3b46bd5e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:51.777176 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342bd19-ce5f-40d4-9f54-af3b46bd5e7c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:51.777640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342bd19-ce5f-40d4-9f54-af3b46bd5e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:51.778118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7342bd19-ce5f-40d4-9f54-af3b46bd5e7c None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:51.779591 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342bd19-ce5f-40d4-9f54-af3b46bd5e7c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:51.780138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342bd19-ce5f-40d4-9f54-af3b46bd5e7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:51.782349 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7342bd19-ce5f-40d4-9f54-af3b46bd5e7c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:51.782845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:51.804599 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 94/193] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:43:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:51.805920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342bd19-ce5f-40d4-9f54-af3b46bd5e7c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:51.863106 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6b201-cd4a-4ea5-9a4e-fc48bb5cedd8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:51.909585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277091e-f164-4119-ba75-76e5ab1a962a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:51.910657 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277091e-f164-4119-ba75-76e5ab1a962a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:51.911146 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277091e-f164-4119-ba75-76e5ab1a962a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:51.911525 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3277091e-f164-4119-ba75-76e5ab1a962a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:51.936174 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:51.940602 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7342bd19-ce5f-40d4-9f54-af3b46bd5e7c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:51.991021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fe408-9dd6-4d9e-9ebd-0e1b61fb7429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:51.991776 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fe408-9dd6-4d9e-9ebd-0e1b61fb7429 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:51.992063 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fe408-9dd6-4d9e-9ebd-0e1b61fb7429 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:51.992342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2fe408-9dd6-4d9e-9ebd-0e1b61fb7429 None admin] PATH_INFO: `/v3/users/136255de56114dec8740f51546436962` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:51.993503 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fe408-9dd6-4d9e-9ebd-0e1b61fb7429 None admin] RBAC: Authorizing `identity:get_user(user_id=136255de56114dec8740f51546436962)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:51.995277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2fe408-9dd6-4d9e-9ebd-0e1b61fb7429 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:52.002518 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 95/194] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:51 2022] GET /identity/v3/users/136255de56114dec8740f51546436962 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:52.002664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277091e-f164-4119-ba75-76e5ab1a962a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:52.002664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3277091e-f164-4119-ba75-76e5ab1a962a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:52.005820 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fe408-9dd6-4d9e-9ebd-0e1b61fb7429 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:52.066736 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:52.145342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2fe408-9dd6-4d9e-9ebd-0e1b61fb7429 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:52.149422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 100/195] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:43:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:43:52.154513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277091e-f164-4119-ba75-76e5ab1a962a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:52.189479 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17891e3-0306-48de-8bba-b94f2974325f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:52.189479 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17891e3-0306-48de-8bba-b94f2974325f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:52.189846 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17891e3-0306-48de-8bba-b94f2974325f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:52.190237 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17891e3-0306-48de-8bba-b94f2974325f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:52.237656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:52.285853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17891e3-0306-48de-8bba-b94f2974325f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:52.286406 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3277091e-f164-4119-ba75-76e5ab1a962a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:52.287603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17891e3-0306-48de-8bba-b94f2974325f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:52.327289 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd97286-e36a-40f5-a1a2-c3b6e23eed90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:52.328472 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd97286-e36a-40f5-a1a2-c3b6e23eed90 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:52.328910 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd97286-e36a-40f5-a1a2-c3b6e23eed90 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:52.329679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd97286-e36a-40f5-a1a2-c3b6e23eed90 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:52.331258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd97286-e36a-40f5-a1a2-c3b6e23eed90 None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:52.333141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd97286-e36a-40f5-a1a2-c3b6e23eed90 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:52.335414 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 101/196] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:52 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:52.340896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd97286-e36a-40f5-a1a2-c3b6e23eed90 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:52.363055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:52.431822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 96/197] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:43:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 17:43:52.438017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17891e3-0306-48de-8bba-b94f2974325f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:52.538659 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd97286-e36a-40f5-a1a2-c3b6e23eed90 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:52.578516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf1ec9-1633-463c-8cec-49c334c3d729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:52.579148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf1ec9-1633-463c-8cec-49c334c3d729 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:52.582952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf1ec9-1633-463c-8cec-49c334c3d729 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:52.582952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf1ec9-1633-463c-8cec-49c334c3d729 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:52.582952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf1ec9-1633-463c-8cec-49c334c3d729 None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=887965a382184db9958ad175261f03a1, user.id=136255de56114dec8740f51546436962) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:52.582952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf1ec9-1633-463c-8cec-49c334c3d729 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:52.582952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf1ec9-1633-463c-8cec-49c334c3d729 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:52.590024 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 102/198] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:43:52 2022] GET /identity/v3/role_assignments?user.id=136255de56114dec8740f51546436962&scope.project.id=887965a382184db9958ad175261f03a1&role.id=07626899472b4e32adc0c71ce796a25d => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:52.592109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:52.672120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17891e3-0306-48de-8bba-b94f2974325f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:52.701139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f163b7-83db-4a20-92c6-1d59304972c7 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:52.701764 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f163b7-83db-4a20-92c6-1d59304972c7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:52.702257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f163b7-83db-4a20-92c6-1d59304972c7 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:52.704246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 103/199] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:43:52.711259 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1e0ae-4a1e-4e57-b289-0a6ae4d6b235 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:52.712056 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1e0ae-4a1e-4e57-b289-0a6ae4d6b235 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:52.712531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1e0ae-4a1e-4e57-b289-0a6ae4d6b235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:52.733997 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-11b1e0ae-4a1e-4e57-b289-0a6ae4d6b235 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:52.736253 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-11b1e0ae-4a1e-4e57-b289-0a6ae4d6b235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:52.752391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500770ae-85c3-4ffe-a744-b8f8267b8508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:52.755768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500770ae-85c3-4ffe-a744-b8f8267b8508 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:52.756362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500770ae-85c3-4ffe-a744-b8f8267b8508 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:52.757795 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500770ae-85c3-4ffe-a744-b8f8267b8508 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:52.760274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500770ae-85c3-4ffe-a744-b8f8267b8508 None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:52.762171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500770ae-85c3-4ffe-a744-b8f8267b8508 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:52.765812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 97/200] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:52 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:52.772930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500770ae-85c3-4ffe-a744-b8f8267b8508 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:52.859803 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 104/201] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:43:52.938232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500770ae-85c3-4ffe-a744-b8f8267b8508 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:52.989145 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90181dfa-e276-473b-92a4-76a9c4505642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:52.989856 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90181dfa-e276-473b-92a4-76a9c4505642 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:52.990157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90181dfa-e276-473b-92a4-76a9c4505642 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:52.990435 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90181dfa-e276-473b-92a4-76a9c4505642 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:52.991141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90181dfa-e276-473b-92a4-76a9c4505642 None admin] RBAC: Adding query filter params (role.id=30bb990dfe5c45fbbfed21c08c7859b4, scope.project.id=887965a382184db9958ad175261f03a1, user.id=136255de56114dec8740f51546436962) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:52.991449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90181dfa-e276-473b-92a4-76a9c4505642 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:52.992965 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90181dfa-e276-473b-92a4-76a9c4505642 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:52.995974 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:52.998774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 98/202] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:43:52 2022] GET /identity/v3/role_assignments?user.id=136255de56114dec8740f51546436962&scope.project.id=887965a382184db9958ad175261f03a1&role.id=30bb990dfe5c45fbbfed21c08c7859b4 => generated 259 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:53.153601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1e0ae-4a1e-4e57-b289-0a6ae4d6b235 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:53.156977 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:53.291967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1e0ae-4a1e-4e57-b289-0a6ae4d6b235 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:53.357661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83e354-aa84-4477-9e4e-5215aa29b31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:53.358310 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83e354-aa84-4477-9e4e-5215aa29b31b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:53.358586 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83e354-aa84-4477-9e4e-5215aa29b31b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:53.358855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83e354-aa84-4477-9e4e-5215aa29b31b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:53.374070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b83e354-aa84-4477-9e4e-5215aa29b31b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-9b83e354-aa84-4477-9e4e-5215aa29b31b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:53.378922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:53.381144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:53.382247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:53.382247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:53.382247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:53.382247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:53.382247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:53.382247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:53.382247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:53.382247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:53.382247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:53.382247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:53.382247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:43:53.382247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:43:53.382247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 105/203] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:53.386405 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90181dfa-e276-473b-92a4-76a9c4505642 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:53.524022 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90181dfa-e276-473b-92a4-76a9c4505642 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:53.559031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade0cc5-ca05-42d7-9d3a-cfa9fbc2ca60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:53.559722 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade0cc5-ca05-42d7-9d3a-cfa9fbc2ca60 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:53.560068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade0cc5-ca05-42d7-9d3a-cfa9fbc2ca60 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:53.560390 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade0cc5-ca05-42d7-9d3a-cfa9fbc2ca60 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:53.561115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade0cc5-ca05-42d7-9d3a-cfa9fbc2ca60 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:53.561419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade0cc5-ca05-42d7-9d3a-cfa9fbc2ca60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:53.562789 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aade0cc5-ca05-42d7-9d3a-cfa9fbc2ca60 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:53.572507 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 99/204] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:43:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:53.578108 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83e354-aa84-4477-9e4e-5215aa29b31b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:53.647919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:53.698193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83e354-aa84-4477-9e4e-5215aa29b31b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:53.738920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a42206-d958-4166-a986-d7c93479da92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:53.739552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a42206-d958-4166-a986-d7c93479da92 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:53.739860 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a42206-d958-4166-a986-d7c93479da92 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:53.740131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a42206-d958-4166-a986-d7c93479da92 None admin] PATH_INFO: `/v3/users/732dc764ad1742e3935aabb4dce1ae5d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:53.741264 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a42206-d958-4166-a986-d7c93479da92 None admin] RBAC: Authorizing `identity:get_user(user_id=732dc764ad1742e3935aabb4dce1ae5d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:53.742691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a42206-d958-4166-a986-d7c93479da92 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:53.744579 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 106/205] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:53 2022] GET /identity/v3/users/732dc764ad1742e3935aabb4dce1ae5d => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:53.750096 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade0cc5-ca05-42d7-9d3a-cfa9fbc2ca60 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:53.813998 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:53.856527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade0cc5-ca05-42d7-9d3a-cfa9fbc2ca60 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:53.892159 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7aab7-ea96-46a0-8d9b-e71ac6cdd190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:53.892773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7aab7-ea96-46a0-8d9b-e71ac6cdd190 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:53.893052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7aab7-ea96-46a0-8d9b-e71ac6cdd190 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:53.893347 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e7aab7-ea96-46a0-8d9b-e71ac6cdd190 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:53.986376 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7aab7-ea96-46a0-8d9b-e71ac6cdd190 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:53.987885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e7aab7-ea96-46a0-8d9b-e71ac6cdd190 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:54.075357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:54.121880 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 100/206] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:43:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:43:54.127463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a42206-d958-4166-a986-d7c93479da92 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:54.264821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a42206-d958-4166-a986-d7c93479da92 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:54.305067 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:54.310432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756ce70a-1910-4d04-876e-10d7596b823b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:54.311072 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ce70a-1910-4d04-876e-10d7596b823b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:54.311393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ce70a-1910-4d04-876e-10d7596b823b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:54.311680 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756ce70a-1910-4d04-876e-10d7596b823b None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:54.316339 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ce70a-1910-4d04-876e-10d7596b823b None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:54.318445 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756ce70a-1910-4d04-876e-10d7596b823b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:54.320273 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 107/207] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:54 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:54.326746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7aab7-ea96-46a0-8d9b-e71ac6cdd190 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:54.445519 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e7aab7-ea96-46a0-8d9b-e71ac6cdd190 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:54.490566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b48145-79a0-4e66-97e7-f2497d1065d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:54.491179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b48145-79a0-4e66-97e7-f2497d1065d1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:54.491512 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b48145-79a0-4e66-97e7-f2497d1065d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:54.491825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b48145-79a0-4e66-97e7-f2497d1065d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:54.492546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b48145-79a0-4e66-97e7-f2497d1065d1 None admin] RBAC: Adding query filter params (role.id=6a413519e66945239f17395ca12293c2, scope.project.id=25808aef0f69482dbf49592f737cff01, user.id=732dc764ad1742e3935aabb4dce1ae5d) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:54.492850 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b48145-79a0-4e66-97e7-f2497d1065d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:54.494405 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b48145-79a0-4e66-97e7-f2497d1065d1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:54.500513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 101/208] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:43:54 2022] GET /identity/v3/role_assignments?user.id=732dc764ad1742e3935aabb4dce1ae5d&scope.project.id=25808aef0f69482dbf49592f737cff01&role.id=6a413519e66945239f17395ca12293c2 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:54.787705 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec9816-2b9f-4fb4-87ad-7dd723db7021 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:54.788209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec9816-2b9f-4fb4-87ad-7dd723db7021 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:54.788209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ec9816-2b9f-4fb4-87ad-7dd723db7021 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:54.789684 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 108/209] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:54.795182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1123ed-5d10-493b-bcfc-5be5179f0621 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:54.795572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1123ed-5d10-493b-bcfc-5be5179f0621 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:54.795758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1123ed-5d10-493b-bcfc-5be5179f0621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:54.814203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1b1123ed-5d10-493b-bcfc-5be5179f0621 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:54.815379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1b1123ed-5d10-493b-bcfc-5be5179f0621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:54.834772 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:54.876931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3720b-3ee9-4245-b777-2fa274a44069 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:54.877211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3720b-3ee9-4245-b777-2fa274a44069 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:54.877484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3720b-3ee9-4245-b777-2fa274a44069 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:54.878844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 109/210] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:43:54.886815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4b06d-2bf0-4563-8330-74a508ace322 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:54.886815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4b06d-2bf0-4563-8330-74a508ace322 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:54.886815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c4b06d-2bf0-4563-8330-74a508ace322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:54.890181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 102/211] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:54.907901 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-31c4b06d-2bf0-4563-8330-74a508ace322 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:54.909132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-31c4b06d-2bf0-4563-8330-74a508ace322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:54.989340 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 110/212] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:43:55.178007 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1123ed-5d10-493b-bcfc-5be5179f0621 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:55.285238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1123ed-5d10-493b-bcfc-5be5179f0621 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:55.302378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4b06d-2bf0-4563-8330-74a508ace322 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:55.325033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6bc34-1dae-4cb8-90f9-7f1b83ef5ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:55.325898 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6bc34-1dae-4cb8-90f9-7f1b83ef5ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:55.326005 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6bc34-1dae-4cb8-90f9-7f1b83ef5ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:55.326261 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f6bc34-1dae-4cb8-90f9-7f1b83ef5ce5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:55.337121 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f6bc34-1dae-4cb8-90f9-7f1b83ef5ce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c6f6bc34-1dae-4cb8-90f9-7f1b83ef5ce5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:55.342206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:55.343040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:55.343824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:55.343824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:55.343824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:55.343824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:55.343824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:55.343824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:55.343824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:55.343824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:55.343824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:55.343824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:55.343824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:43:55.343824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:43:55.343824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 103/213] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:55.351345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6bc34-1dae-4cb8-90f9-7f1b83ef5ce5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:55.369428 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:55.431975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c4b06d-2bf0-4563-8330-74a508ace322 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:55.460515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f6bc34-1dae-4cb8-90f9-7f1b83ef5ce5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:55.465456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:55.473870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d9a72d-cb36-4379-be46-bbcd1d83ef8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:55.474453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9a72d-cb36-4379-be46-bbcd1d83ef8a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:55.474913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9a72d-cb36-4379-be46-bbcd1d83ef8a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:55.475192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d9a72d-cb36-4379-be46-bbcd1d83ef8a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:55.476743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9a72d-cb36-4379-be46-bbcd1d83ef8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:55.478201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d9a72d-cb36-4379-be46-bbcd1d83ef8a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:55.479922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 111/214] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:43:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:55.481107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98c30b-92cc-40c8-9bd3-943293644441 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:55.481394 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98c30b-92cc-40c8-9bd3-943293644441 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:55.481671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c98c30b-92cc-40c8-9bd3-943293644441 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:55.483686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 112/215] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:55.485126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98c30b-92cc-40c8-9bd3-943293644441 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:55.501621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b407b756-6eb4-4636-9d44-a369acee2c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:55.502469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407b756-6eb4-4636-9d44-a369acee2c12 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:55.502758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407b756-6eb4-4636-9d44-a369acee2c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:55.503031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b407b756-6eb4-4636-9d44-a369acee2c12 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:55.503732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407b756-6eb4-4636-9d44-a369acee2c12 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:55.504582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407b756-6eb4-4636-9d44-a369acee2c12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:55.506390 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b407b756-6eb4-4636-9d44-a369acee2c12 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:55.515477 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 104/216] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:43:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:55.521856 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575430e5-5ff9-4ec3-89ac-58fa1b4e46e0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:55.522000 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575430e5-5ff9-4ec3-89ac-58fa1b4e46e0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:55.522276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575430e5-5ff9-4ec3-89ac-58fa1b4e46e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:55.542171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-575430e5-5ff9-4ec3-89ac-58fa1b4e46e0 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:55.543362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-575430e5-5ff9-4ec3-89ac-58fa1b4e46e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:55.599047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c98c30b-92cc-40c8-9bd3-943293644441 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:55.639677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 105/217] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:55.639677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575430e5-5ff9-4ec3-89ac-58fa1b4e46e0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:55.642433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:55.651755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26820f-ea46-4599-840e-b6747fd0a968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:55.652375 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26820f-ea46-4599-840e-b6747fd0a968 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:55.652663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26820f-ea46-4599-840e-b6747fd0a968 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:55.652976 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26820f-ea46-4599-840e-b6747fd0a968 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:55.653694 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26820f-ea46-4599-840e-b6747fd0a968 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:55.655190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26820f-ea46-4599-840e-b6747fd0a968 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:55.748699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575430e5-5ff9-4ec3-89ac-58fa1b4e46e0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:55.755971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 113/218] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:43:55 2022] POST /identity/v3/users => generated 297 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:43:55.761138 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581c5b4-5d86-4e0c-9398-fa4c6001cf9f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:55.761420 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581c5b4-5d86-4e0c-9398-fa4c6001cf9f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:55.761698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e581c5b4-5d86-4e0c-9398-fa4c6001cf9f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:55.763267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 114/219] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:55.770195 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a614c-934a-46b2-aae2-fdb5d5717847 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:55.771013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a614c-934a-46b2-aae2-fdb5d5717847 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:55.771690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a614c-934a-46b2-aae2-fdb5d5717847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:55.792836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ab0a614c-934a-46b2-aae2-fdb5d5717847 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:55.794851 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ab0a614c-934a-46b2-aae2-fdb5d5717847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:55.796649 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300822a5-7405-4767-b51d-3cd4a215df25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:55.797360 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300822a5-7405-4767-b51d-3cd4a215df25 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:55.797647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300822a5-7405-4767-b51d-3cd4a215df25 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:55.797996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300822a5-7405-4767-b51d-3cd4a215df25 None admin] PATH_INFO: `/v3/users/4b32ea2da2eb41e0bd5d9a8cd9d690bb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:55.815271 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300822a5-7405-4767-b51d-3cd4a215df25 None admin] RBAC: Authorizing `identity:get_user(user_id=4b32ea2da2eb41e0bd5d9a8cd9d690bb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:55.817149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300822a5-7405-4767-b51d-3cd4a215df25 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:55.819550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 106/220] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:55 2022] GET /identity/v3/users/4b32ea2da2eb41e0bd5d9a8cd9d690bb => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:55.825299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300822a5-7405-4767-b51d-3cd4a215df25 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:55.854493 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:55.874297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 115/221] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:43:55.901281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:55.919823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a614c-934a-46b2-aae2-fdb5d5717847 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:55.951600 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300822a5-7405-4767-b51d-3cd4a215df25 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:56.008793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d3ee8-945d-4e77-8fdf-688e8251478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:56.009317 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d3ee8-945d-4e77-8fdf-688e8251478b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:56.009603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d3ee8-945d-4e77-8fdf-688e8251478b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:56.009879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d3ee8-945d-4e77-8fdf-688e8251478b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:56.064422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a614c-934a-46b2-aae2-fdb5d5717847 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:56.093153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d3ee8-945d-4e77-8fdf-688e8251478b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:56.094634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d3ee8-945d-4e77-8fdf-688e8251478b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:56.111386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95af7a-3772-4883-8fd9-2219e59c44d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:56.112613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95af7a-3772-4883-8fd9-2219e59c44d0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:56.112886 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95af7a-3772-4883-8fd9-2219e59c44d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:56.114162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b95af7a-3772-4883-8fd9-2219e59c44d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:56.128245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b95af7a-3772-4883-8fd9-2219e59c44d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-7b95af7a-3772-4883-8fd9-2219e59c44d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:56.134322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:56.135206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:56.135994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:56.135994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:56.135994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:56.135994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:56.135994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:56.135994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:56.135994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:56.135994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:56.135994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:56.135994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:56.135994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:43:56.135994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:43:56.137059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 116/222] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:43:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:43:56.148156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95af7a-3772-4883-8fd9-2219e59c44d0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:56.176518 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:56.210698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:56.228055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 107/223] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:43:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 17:43:56.228055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d3ee8-945d-4e77-8fdf-688e8251478b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:56.277902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b95af7a-3772-4883-8fd9-2219e59c44d0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:56.322180 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7dbf4-18a8-4c23-9f48-7b2f3f5d577f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:56.323044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7dbf4-18a8-4c23-9f48-7b2f3f5d577f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:56.323044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7dbf4-18a8-4c23-9f48-7b2f3f5d577f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:56.323329 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7dbf4-18a8-4c23-9f48-7b2f3f5d577f None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:56.324082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7dbf4-18a8-4c23-9f48-7b2f3f5d577f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:56.324391 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7dbf4-18a8-4c23-9f48-7b2f3f5d577f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:56.325788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7dbf4-18a8-4c23-9f48-7b2f3f5d577f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:56.336518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 117/224] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:56.337268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7dbf4-18a8-4c23-9f48-7b2f3f5d577f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:56.353928 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d3ee8-945d-4e77-8fdf-688e8251478b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:56.403406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ddf9f-c4b1-411b-b986-e8b7423f3a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:56.404487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ddf9f-c4b1-411b-b986-e8b7423f3a32 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:56.405552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ddf9f-c4b1-411b-b986-e8b7423f3a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:56.406166 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ddf9f-c4b1-411b-b986-e8b7423f3a32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:56.421041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ddf9f-c4b1-411b-b986-e8b7423f3a32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b18ddf9f-c4b1-411b-b986-e8b7423f3a32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:56.428275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:56.429120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:56.429871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:56.429871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:56.429871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:56.429871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:56.429871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:56.429871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:56.429871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:56.429871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:56.429871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:56.429871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:56.429871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:43:56.429871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:43:56.432907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 108/225] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:43:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:56.433879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ddf9f-c4b1-411b-b986-e8b7423f3a32 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:56.464325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:56.467298 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7dbf4-18a8-4c23-9f48-7b2f3f5d577f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:56.477491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:56.510965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e49a51-d830-4ba1-80a5-13ae6c463fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:56.512488 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e49a51-d830-4ba1-80a5-13ae6c463fdc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:56.513249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e49a51-d830-4ba1-80a5-13ae6c463fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:56.513733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e49a51-d830-4ba1-80a5-13ae6c463fdc None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:56.515396 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e49a51-d830-4ba1-80a5-13ae6c463fdc None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:56.517091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e49a51-d830-4ba1-80a5-13ae6c463fdc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:56.519164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 118/226] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:56 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:56.520011 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e49a51-d830-4ba1-80a5-13ae6c463fdc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:56.567268 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ddf9f-c4b1-411b-b986-e8b7423f3a32 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:56.612197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b3198-3bba-407c-a5d1-cf13241634cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:56.612988 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b3198-3bba-407c-a5d1-cf13241634cb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:56.613923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b3198-3bba-407c-a5d1-cf13241634cb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:56.614261 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b3198-3bba-407c-a5d1-cf13241634cb None admin] PATH_INFO: `/v3/users/136255de56114dec8740f51546436962` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:56.616079 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b3198-3bba-407c-a5d1-cf13241634cb None admin] RBAC: Authorizing `identity:get_user(user_id=136255de56114dec8740f51546436962)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:56.618345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b3198-3bba-407c-a5d1-cf13241634cb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:56.620402 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 109/227] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:56 2022] GET /identity/v3/users/136255de56114dec8740f51546436962 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:56.621710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b3198-3bba-407c-a5d1-cf13241634cb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:56.643484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e49a51-d830-4ba1-80a5-13ae6c463fdc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:56.686698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87066034-ac42-4d38-a01c-d9c0dd892fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:56.687804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87066034-ac42-4d38-a01c-d9c0dd892fdd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:56.688543 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87066034-ac42-4d38-a01c-d9c0dd892fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:56.688988 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87066034-ac42-4d38-a01c-d9c0dd892fdd None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:56.689817 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87066034-ac42-4d38-a01c-d9c0dd892fdd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:56.690312 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87066034-ac42-4d38-a01c-d9c0dd892fdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:56.693038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87066034-ac42-4d38-a01c-d9c0dd892fdd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:56.703243 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 119/228] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:43:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:56.704387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87066034-ac42-4d38-a01c-d9c0dd892fdd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:56.743120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:56.749432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:56.755301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b3198-3bba-407c-a5d1-cf13241634cb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:56.796941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d40d9c-ea74-4e11-bd4e-db8e7880c927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:56.798488 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d40d9c-ea74-4e11-bd4e-db8e7880c927 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:56.799031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d40d9c-ea74-4e11-bd4e-db8e7880c927 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:56.799525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d40d9c-ea74-4e11-bd4e-db8e7880c927 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:56.800481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d40d9c-ea74-4e11-bd4e-db8e7880c927 None admin] RBAC: Adding query filter params (role.id=1c983c7e32e44c61be3996c28399e617, scope.project.id=887965a382184db9958ad175261f03a1, user.id=4b32ea2da2eb41e0bd5d9a8cd9d690bb) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:56.801021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d40d9c-ea74-4e11-bd4e-db8e7880c927 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:56.802665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d40d9c-ea74-4e11-bd4e-db8e7880c927 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:56.809237 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 110/229] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:43:56 2022] GET /identity/v3/role_assignments?user.id=4b32ea2da2eb41e0bd5d9a8cd9d690bb&scope.project.id=887965a382184db9958ad175261f03a1&role.id=1c983c7e32e44c61be3996c28399e617 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:56.810286 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d40d9c-ea74-4e11-bd4e-db8e7880c927 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:56.898019 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87066034-ac42-4d38-a01c-d9c0dd892fdd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:56.940251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d40d9c-ea74-4e11-bd4e-db8e7880c927 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:56.986972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f5a47-8dee-4d2d-a592-1b4e0074ec66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:56.988061 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f5a47-8dee-4d2d-a592-1b4e0074ec66 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:56.989339 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f5a47-8dee-4d2d-a592-1b4e0074ec66 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:56.989811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f5a47-8dee-4d2d-a592-1b4e0074ec66 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:56.993313 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f5a47-8dee-4d2d-a592-1b4e0074ec66 None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:56.994257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f5a47-8dee-4d2d-a592-1b4e0074ec66 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:57.000982 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 120/230] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:56 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:57.001876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f5a47-8dee-4d2d-a592-1b4e0074ec66 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:57.018939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb72746-4fe3-436b-a1f2-ed529b23459b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:57.019869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb72746-4fe3-436b-a1f2-ed529b23459b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:57.020077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb72746-4fe3-436b-a1f2-ed529b23459b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:57.020303 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb72746-4fe3-436b-a1f2-ed529b23459b None admin] PATH_INFO: `/v3/users/136255de56114dec8740f51546436962` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:57.021744 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb72746-4fe3-436b-a1f2-ed529b23459b None admin] RBAC: Authorizing `identity:get_user(user_id=136255de56114dec8740f51546436962)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:57.023195 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb72746-4fe3-436b-a1f2-ed529b23459b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:57.024806 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 111/231] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:56 2022] GET /identity/v3/users/136255de56114dec8740f51546436962 => generated 283 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:57.034434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb72746-4fe3-436b-a1f2-ed529b23459b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:57.082827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:57.182987 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f5a47-8dee-4d2d-a592-1b4e0074ec66 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:57.187057 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:57.221387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb72746-4fe3-436b-a1f2-ed529b23459b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:57.247813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b35e8-4157-47f5-85e7-a4e047af72ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:57.248867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b35e8-4157-47f5-85e7-a4e047af72ab None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:57.249471 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b35e8-4157-47f5-85e7-a4e047af72ab None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:57.249948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b35e8-4157-47f5-85e7-a4e047af72ab None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1/users/136255de56114dec8740f51546436962/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:57.253052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b35e8-4157-47f5-85e7-a4e047af72ab None admin] RBAC: Authorizing `identity:create_grant(project_id=887965a382184db9958ad175261f03a1, user_id=136255de56114dec8740f51546436962, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:57.255390 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b35e8-4157-47f5-85e7-a4e047af72ab None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:57.280040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c056a6-99ad-4b25-9d03-e27d4571c4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:57.284048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c056a6-99ad-4b25-9d03-e27d4571c4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:57.284048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c056a6-99ad-4b25-9d03-e27d4571c4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:57.284048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c056a6-99ad-4b25-9d03-e27d4571c4c6 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:57.284369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c056a6-99ad-4b25-9d03-e27d4571c4c6 None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:57.286372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c056a6-99ad-4b25-9d03-e27d4571c4c6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:57.288396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 112/232] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:43:57 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:43:57.293881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c056a6-99ad-4b25-9d03-e27d4571c4c6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:57.355777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 121/233] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:43:57 2022] PUT /identity/v3/projects/887965a382184db9958ad175261f03a1/users/136255de56114dec8740f51546436962/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:43:57.415825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:57.425618 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c056a6-99ad-4b25-9d03-e27d4571c4c6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:57.464125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba35f5-86cc-4c4b-963c-f24175ec6173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:57.464712 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba35f5-86cc-4c4b-963c-f24175ec6173 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:57.464984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba35f5-86cc-4c4b-963c-f24175ec6173 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:57.465253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba35f5-86cc-4c4b-963c-f24175ec6173 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1/users/136255de56114dec8740f51546436962/roles/30bb990dfe5c45fbbfed21c08c7859b4` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:57.467598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba35f5-86cc-4c4b-963c-f24175ec6173 None admin] RBAC: Authorizing `identity:create_grant(project_id=887965a382184db9958ad175261f03a1, user_id=136255de56114dec8740f51546436962, role_id=30bb990dfe5c45fbbfed21c08c7859b4)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:57.469169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba35f5-86cc-4c4b-963c-f24175ec6173 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:57.550140 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 113/234] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:43:57 2022] PUT /identity/v3/projects/887965a382184db9958ad175261f03a1/users/136255de56114dec8740f51546436962/roles/30bb990dfe5c45fbbfed21c08c7859b4 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:43:58.353904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25612f-0f07-4300-a86c-2f5e4ef088b0 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:58.354430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25612f-0f07-4300-a86c-2f5e4ef088b0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:58.354430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e25612f-0f07-4300-a86c-2f5e4ef088b0 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:58.355971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 122/235] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:58.361593 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17031e8e-b38c-49e4-b36f-41b275abd736 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:58.362091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17031e8e-b38c-49e4-b36f-41b275abd736 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:58.362162 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17031e8e-b38c-49e4-b36f-41b275abd736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:58.388750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-17031e8e-b38c-49e4-b36f-41b275abd736 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:58.390240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-17031e8e-b38c-49e4-b36f-41b275abd736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:58.484329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 114/236] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:58.531475 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a83828-ba7c-4708-b7eb-34cc6b144277 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:58.531795 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a83828-ba7c-4708-b7eb-34cc6b144277 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:58.532063 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a83828-ba7c-4708-b7eb-34cc6b144277 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:58.533560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 123/237] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:58.539714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43a0c5-7d53-47e1-8c83-78fed5d43053 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:58.540009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43a0c5-7d53-47e1-8c83-78fed5d43053 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:58.540286 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d43a0c5-7d53-47e1-8c83-78fed5d43053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:58.565413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5d43a0c5-7d53-47e1-8c83-78fed5d43053 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:58.567327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5d43a0c5-7d53-47e1-8c83-78fed5d43053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:58.648640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 115/238] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:58.950061 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a83828-ba7c-4708-b7eb-34cc6b144277 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:59.061135 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43a0c5-7d53-47e1-8c83-78fed5d43053 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:59.070605 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a83828-ba7c-4708-b7eb-34cc6b144277 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:59.095771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:59.121788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a282d-3f82-4678-9d46-7d464f071525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:59.122486 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a282d-3f82-4678-9d46-7d464f071525 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:59.122769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a282d-3f82-4678-9d46-7d464f071525 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:59.123097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a282d-3f82-4678-9d46-7d464f071525 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:59.147745 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a282d-3f82-4678-9d46-7d464f071525 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e75a282d-3f82-4678-9d46-7d464f071525 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:43:59.153608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:43:59.154492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:43:59.155277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:43:59.155277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:43:59.155277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:43:59.155277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:43:59.155277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:43:59.155277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:43:59.155277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:43:59.155277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:43:59.155277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:43:59.155277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:43:59.155277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:43:59.155277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:43:59.155277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 124/239] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:43:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:43:59.162186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a282d-3f82-4678-9d46-7d464f071525 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:59.213440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:59.263549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d43a0c5-7d53-47e1-8c83-78fed5d43053 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:59.306813 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e16d9-730b-46ef-9bc0-cec89983d7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:59.307441 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e16d9-730b-46ef-9bc0-cec89983d7cf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:59.307815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e16d9-730b-46ef-9bc0-cec89983d7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:59.308087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e16d9-730b-46ef-9bc0-cec89983d7cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:59.309179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e16d9-730b-46ef-9bc0-cec89983d7cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:59.315680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e16d9-730b-46ef-9bc0-cec89983d7cf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:59.315680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 116/240] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:43:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:59.319612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e16d9-730b-46ef-9bc0-cec89983d7cf None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:59.406588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a282d-3f82-4678-9d46-7d464f071525 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:59.432176 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:59.468676 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e16d9-730b-46ef-9bc0-cec89983d7cf None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:59.480339 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65027713-3b73-423e-acf3-386f6f74b0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:59.480957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65027713-3b73-423e-acf3-386f6f74b0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:59.481235 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65027713-3b73-423e-acf3-386f6f74b0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:59.481504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65027713-3b73-423e-acf3-386f6f74b0a0 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:59.482178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65027713-3b73-423e-acf3-386f6f74b0a0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:43:59.482478 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65027713-3b73-423e-acf3-386f6f74b0a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:59.483878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65027713-3b73-423e-acf3-386f6f74b0a0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:59.502287 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 125/241] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:43:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:59.506242 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66976c54-3c9e-4844-9062-a2ed1fdfc8b7 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:59.506670 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66976c54-3c9e-4844-9062-a2ed1fdfc8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:59.507088 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66976c54-3c9e-4844-9062-a2ed1fdfc8b7 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:59.508771 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 126/242] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:43:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:43:59.509472 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66976c54-3c9e-4844-9062-a2ed1fdfc8b7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:59.535960 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1a7a5-c26e-423d-b067-3b4e2d2352ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:59.536581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1a7a5-c26e-423d-b067-3b4e2d2352ee None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:59.536927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1a7a5-c26e-423d-b067-3b4e2d2352ee None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:59.537206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1a7a5-c26e-423d-b067-3b4e2d2352ee None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:59.538753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1a7a5-c26e-423d-b067-3b4e2d2352ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:59.540110 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1a7a5-c26e-423d-b067-3b4e2d2352ee None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:59.609085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:43:59.626077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66976c54-3c9e-4844-9062-a2ed1fdfc8b7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:59.659671 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 117/243] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:43:59 2022] POST /identity/v3/users => generated 293 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:43:59.659671 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851ea52-7b54-46ee-9948-9ae8f8c76b94 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:59.659671 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851ea52-7b54-46ee-9948-9ae8f8c76b94 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:59.659671 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8851ea52-7b54-46ee-9948-9ae8f8c76b94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:59.674550 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda489bc-2fed-456e-852e-a55edf075e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:59.675558 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda489bc-2fed-456e-852e-a55edf075e44 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:59.676950 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda489bc-2fed-456e-852e-a55edf075e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:59.677899 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda489bc-2fed-456e-852e-a55edf075e44 None admin] PATH_INFO: `/v3/users/74ea797808eb402eb58427ea8e599a00` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:59.681403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8851ea52-7b54-46ee-9948-9ae8f8c76b94 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:43:59.681403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8851ea52-7b54-46ee-9948-9ae8f8c76b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:43:59.711588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda489bc-2fed-456e-852e-a55edf075e44 None admin] RBAC: Authorizing `identity:get_user(user_id=74ea797808eb402eb58427ea8e599a00)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:43:59.716430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda489bc-2fed-456e-852e-a55edf075e44 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:43:59.718324 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 127/244] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:43:59 2022] GET /identity/v3/users/74ea797808eb402eb58427ea8e599a00 => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:43:59.725793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda489bc-2fed-456e-852e-a55edf075e44 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:43:59.869741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 118/245] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:43:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:43:59.874050 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda489bc-2fed-456e-852e-a55edf075e44 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:43:59.914277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0988cb1-30a4-4e39-b66c-0a5d382fec54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:43:59.914909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0988cb1-30a4-4e39-b66c-0a5d382fec54 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:43:59.915195 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0988cb1-30a4-4e39-b66c-0a5d382fec54 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:43:59.915465 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0988cb1-30a4-4e39-b66c-0a5d382fec54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:43:59.932691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:00.005715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0988cb1-30a4-4e39-b66c-0a5d382fec54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:00.007586 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0988cb1-30a4-4e39-b66c-0a5d382fec54 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:00.036311 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32ca0d-dfd4-4793-ac5c-1fc508cca667 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:00.036652 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32ca0d-dfd4-4793-ac5c-1fc508cca667 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:00.037008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d32ca0d-dfd4-4793-ac5c-1fc508cca667 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:00.038500 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 119/246] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:44:00.044139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbf068-7e5e-4647-885d-7e150f3d5cda None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:00.044422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbf068-7e5e-4647-885d-7e150f3d5cda None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:00.044697 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dbf068-7e5e-4647-885d-7e150f3d5cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:00.070757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e9dbf068-7e5e-4647-885d-7e150f3d5cda None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:00.072144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e9dbf068-7e5e-4647-885d-7e150f3d5cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:00.159222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 128/247] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:43:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:00.166837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:00.167054 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0988cb1-30a4-4e39-b66c-0a5d382fec54 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:00.178899 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 120/248] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:00.204026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbf068-7e5e-4647-885d-7e150f3d5cda None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:00.240649 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:00.316671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0988cb1-30a4-4e39-b66c-0a5d382fec54 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:00.344680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dbf068-7e5e-4647-885d-7e150f3d5cda None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:00.359582 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43ebabf-343c-49b7-a8df-01e2d2da28a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:00.360550 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ebabf-343c-49b7-a8df-01e2d2da28a6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:00.360978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ebabf-343c-49b7-a8df-01e2d2da28a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:00.361254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43ebabf-343c-49b7-a8df-01e2d2da28a6 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:00.362382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ebabf-343c-49b7-a8df-01e2d2da28a6 None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:00.364494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43ebabf-343c-49b7-a8df-01e2d2da28a6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:00.366302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 129/249] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:00 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:00.367389 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f095ca5-e9a5-40b2-8860-e1aab2abe665 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:00.367927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f095ca5-e9a5-40b2-8860-e1aab2abe665 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:00.367927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f095ca5-e9a5-40b2-8860-e1aab2abe665 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:00.369412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 130/250] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:00.374188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f095ca5-e9a5-40b2-8860-e1aab2abe665 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:00.398166 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef744c-e920-426b-9d61-75421b13103e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:00.399099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef744c-e920-426b-9d61-75421b13103e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:00.399375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef744c-e920-426b-9d61-75421b13103e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:00.399665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef744c-e920-426b-9d61-75421b13103e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:00.415476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef744c-e920-426b-9d61-75421b13103e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-dfef744c-e920-426b-9d61-75421b13103e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:00.422745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:00.423620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:00.424486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:00.424486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:00.424486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:00.424486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:00.424486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:00.424486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:00.424486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:00.424486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:00.424486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:00.424486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:00.424486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:00.424486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:00.424486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 121/251] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:00.426726 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b084a-0ff8-4985-a6df-e5d41dc3bb8a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:00.426726 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b084a-0ff8-4985-a6df-e5d41dc3bb8a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:00.427422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8b084a-0ff8-4985-a6df-e5d41dc3bb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:00.450047 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9f8b084a-0ff8-4985-a6df-e5d41dc3bb8a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:00.451778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9f8b084a-0ff8-4985-a6df-e5d41dc3bb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:00.502971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f095ca5-e9a5-40b2-8860-e1aab2abe665 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:00.510858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:00.519113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:00.530320 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 122/252] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:00.535727 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b084a-0ff8-4985-a6df-e5d41dc3bb8a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:00.541432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e9c52-c6e8-4c99-a508-c1ef1b5214e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:00.542048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e9c52-c6e8-4c99-a508-c1ef1b5214e0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:00.542333 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e9c52-c6e8-4c99-a508-c1ef1b5214e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:00.542605 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855e9c52-c6e8-4c99-a508-c1ef1b5214e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:00.543327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e9c52-c6e8-4c99-a508-c1ef1b5214e0 None admin] RBAC: Adding query filter params (role.id=6a413519e66945239f17395ca12293c2, scope.project.id=887965a382184db9958ad175261f03a1, user.id=74ea797808eb402eb58427ea8e599a00) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:00.543630 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e9c52-c6e8-4c99-a508-c1ef1b5214e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:00.545107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855e9c52-c6e8-4c99-a508-c1ef1b5214e0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:00.551720 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 131/253] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:00 2022] GET /identity/v3/role_assignments?user.id=74ea797808eb402eb58427ea8e599a00&scope.project.id=887965a382184db9958ad175261f03a1&role.id=6a413519e66945239f17395ca12293c2 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:00.552782 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e9c52-c6e8-4c99-a508-c1ef1b5214e0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:00.776061 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8b084a-0ff8-4985-a6df-e5d41dc3bb8a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:00.839370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855e9c52-c6e8-4c99-a508-c1ef1b5214e0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:00.860976 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411051b9-ca43-4759-85f5-98ca3ba3334e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:00.861972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411051b9-ca43-4759-85f5-98ca3ba3334e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:00.862286 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411051b9-ca43-4759-85f5-98ca3ba3334e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:00.862566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411051b9-ca43-4759-85f5-98ca3ba3334e None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:00.863326 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411051b9-ca43-4759-85f5-98ca3ba3334e None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:00.864026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411051b9-ca43-4759-85f5-98ca3ba3334e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:00.865511 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411051b9-ca43-4759-85f5-98ca3ba3334e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:00.884971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 123/254] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:00.891104 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411051b9-ca43-4759-85f5-98ca3ba3334e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:00.928810 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06574c0-efd2-4054-9587-0f90249c7172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:00.929462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06574c0-efd2-4054-9587-0f90249c7172 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:00.929804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06574c0-efd2-4054-9587-0f90249c7172 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:00.930083 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06574c0-efd2-4054-9587-0f90249c7172 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:00.955218 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06574c0-efd2-4054-9587-0f90249c7172 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a06574c0-efd2-4054-9587-0f90249c7172 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:00.961233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:00.962297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:00.963269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:00.963269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:00.963269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:00.963269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:00.963269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:00.963269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:00.963269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:00.963269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:00.963269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:00.963269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:00.963269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:00.963269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:00.968861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 132/255] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:44:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 417 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:00.969649 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06574c0-efd2-4054-9587-0f90249c7172 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:00.987704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:01.043345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:01.084646 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411051b9-ca43-4759-85f5-98ca3ba3334e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:01.161066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea645e-9609-4619-8c6f-2ef1a8fa6836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:01.161717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea645e-9609-4619-8c6f-2ef1a8fa6836 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:01.161995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea645e-9609-4619-8c6f-2ef1a8fa6836 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:01.162266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea645e-9609-4619-8c6f-2ef1a8fa6836 None admin] PATH_INFO: `/v3/users/4b32ea2da2eb41e0bd5d9a8cd9d690bb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:01.164660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea645e-9609-4619-8c6f-2ef1a8fa6836 None admin] RBAC: Authorizing `identity:get_user(user_id=4b32ea2da2eb41e0bd5d9a8cd9d690bb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:01.166169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea645e-9609-4619-8c6f-2ef1a8fa6836 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:01.169566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 124/256] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:00 2022] GET /identity/v3/users/4b32ea2da2eb41e0bd5d9a8cd9d690bb => generated 297 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:01.170366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea645e-9609-4619-8c6f-2ef1a8fa6836 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:01.178882 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06574c0-efd2-4054-9587-0f90249c7172 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:01.231023 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb2c33-7f38-4f53-a7fc-aeec9c8cfd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:01.232065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb2c33-7f38-4f53-a7fc-aeec9c8cfd85 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:01.232570 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb2c33-7f38-4f53-a7fc-aeec9c8cfd85 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:01.233044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb2c33-7f38-4f53-a7fc-aeec9c8cfd85 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:01.255354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cb2c33-7f38-4f53-a7fc-aeec9c8cfd85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-59cb2c33-7f38-4f53-a7fc-aeec9c8cfd85 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:01.261182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:01.262036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:01.262802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:01.262802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:01.262802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:01.262802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:01.262802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:01.262802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:01.262802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:01.262802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:01.262802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:01.262802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:01.262802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:01.262802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:01.267411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 133/257] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:44:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:01.270507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb2c33-7f38-4f53-a7fc-aeec9c8cfd85 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:01.354302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea645e-9609-4619-8c6f-2ef1a8fa6836 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:01.373485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:01.401677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac856ad-9781-4260-b075-51aae44c2fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:01.402532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac856ad-9781-4260-b075-51aae44c2fcd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:01.402816 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac856ad-9781-4260-b075-51aae44c2fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:01.403083 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac856ad-9781-4260-b075-51aae44c2fcd None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:01.404646 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac856ad-9781-4260-b075-51aae44c2fcd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:01.404948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac856ad-9781-4260-b075-51aae44c2fcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:01.406364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac856ad-9781-4260-b075-51aae44c2fcd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:01.424148 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 125/258] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:01.424804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac856ad-9781-4260-b075-51aae44c2fcd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:01.453682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:01.465504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb2c33-7f38-4f53-a7fc-aeec9c8cfd85 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:01.515878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a727a3a-2911-4b3a-b504-42898b3126ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:01.516517 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727a3a-2911-4b3a-b504-42898b3126ed None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:01.516777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727a3a-2911-4b3a-b504-42898b3126ed None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:01.517049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727a3a-2911-4b3a-b504-42898b3126ed None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:01.519279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a727a3a-2911-4b3a-b504-42898b3126ed None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:01.524008 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a727a3a-2911-4b3a-b504-42898b3126ed None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:01.526111 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 134/259] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:01 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:01.526695 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a727a3a-2911-4b3a-b504-42898b3126ed None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:01.610426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac856ad-9781-4260-b075-51aae44c2fcd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:01.678336 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ee2e3-c3db-4acb-b463-01fc1cc76f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:01.679000 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ee2e3-c3db-4acb-b463-01fc1cc76f43 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:01.679281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ee2e3-c3db-4acb-b463-01fc1cc76f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:01.679550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4ee2e3-c3db-4acb-b463-01fc1cc76f43 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:01.680235 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ee2e3-c3db-4acb-b463-01fc1cc76f43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:01.680533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ee2e3-c3db-4acb-b463-01fc1cc76f43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:01.682197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4ee2e3-c3db-4acb-b463-01fc1cc76f43 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:01.702675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 126/260] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:01.703484 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ee2e3-c3db-4acb-b463-01fc1cc76f43 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:01.715127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a727a3a-2911-4b3a-b504-42898b3126ed None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:01.776930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:01.777677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3001821-1252-45bc-a324-08bb60720eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:01.778309 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3001821-1252-45bc-a324-08bb60720eac None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:01.778584 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3001821-1252-45bc-a324-08bb60720eac None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:01.778850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3001821-1252-45bc-a324-08bb60720eac None admin] PATH_INFO: `/v3/users/136255de56114dec8740f51546436962` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:01.780574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3001821-1252-45bc-a324-08bb60720eac None admin] RBAC: Authorizing `identity:get_user(user_id=136255de56114dec8740f51546436962)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:01.782013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3001821-1252-45bc-a324-08bb60720eac None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:01.783577 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 135/261] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:01 2022] GET /identity/v3/users/136255de56114dec8740f51546436962 => generated 283 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:01.784343 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3001821-1252-45bc-a324-08bb60720eac None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:01.852928 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:01.879701 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4ee2e3-c3db-4acb-b463-01fc1cc76f43 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:01.925416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3001821-1252-45bc-a324-08bb60720eac None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:01.941703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3487972-4076-429d-92a7-9680b8edb140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:01.942731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3487972-4076-429d-92a7-9680b8edb140 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:01.943017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3487972-4076-429d-92a7-9680b8edb140 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:01.943385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3487972-4076-429d-92a7-9680b8edb140 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1/users/4b32ea2da2eb41e0bd5d9a8cd9d690bb/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:01.949276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3487972-4076-429d-92a7-9680b8edb140 None admin] RBAC: Authorizing `identity:create_grant(project_id=887965a382184db9958ad175261f03a1, user_id=4b32ea2da2eb41e0bd5d9a8cd9d690bb, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:01.950803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3487972-4076-429d-92a7-9680b8edb140 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:01.961199 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831bae9-54f3-44f2-b6c4-82818085ce74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:01.961890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831bae9-54f3-44f2-b6c4-82818085ce74 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:01.962172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831bae9-54f3-44f2-b6c4-82818085ce74 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:01.962445 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6831bae9-54f3-44f2-b6c4-82818085ce74 None admin] PATH_INFO: `/v3/users/136255de56114dec8740f51546436962` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:01.963629 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831bae9-54f3-44f2-b6c4-82818085ce74 None admin] RBAC: Authorizing `identity:get_user(user_id=136255de56114dec8740f51546436962)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:01.965190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6831bae9-54f3-44f2-b6c4-82818085ce74 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:01.967244 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 136/262] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:01 2022] GET /identity/v3/users/136255de56114dec8740f51546436962 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:01.968157 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831bae9-54f3-44f2-b6c4-82818085ce74 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:02.034395 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 127/263] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:44:01 2022] PUT /identity/v3/projects/887965a382184db9958ad175261f03a1/users/4b32ea2da2eb41e0bd5d9a8cd9d690bb/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:02.035770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3487972-4076-429d-92a7-9680b8edb140 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:02.096519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:02.101583 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6831bae9-54f3-44f2-b6c4-82818085ce74 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:02.162505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e68e6-c8a4-4c23-b972-f1466063436a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:02.163118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e68e6-c8a4-4c23-b972-f1466063436a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:02.163622 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e68e6-c8a4-4c23-b972-f1466063436a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:02.163750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14e68e6-c8a4-4c23-b972-f1466063436a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:02.286946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e68e6-c8a4-4c23-b972-f1466063436a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:02.288820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14e68e6-c8a4-4c23-b972-f1466063436a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:02.333702 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:02.341937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3487972-4076-429d-92a7-9680b8edb140 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:02.423912 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 137/264] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:02.424076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:02.430169 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e68e6-c8a4-4c23-b972-f1466063436a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:02.444451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ed126-1198-4009-943a-dd6898924623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:02.445090 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ed126-1198-4009-943a-dd6898924623 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:02.445378 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ed126-1198-4009-943a-dd6898924623 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:02.445653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ed126-1198-4009-943a-dd6898924623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:02.567928 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ed126-1198-4009-943a-dd6898924623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:02.569806 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ed126-1198-4009-943a-dd6898924623 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:02.637898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14e68e6-c8a4-4c23-b972-f1466063436a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:02.727835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a93de-ff86-4391-85f5-928e4adf7942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:02.728829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a93de-ff86-4391-85f5-928e4adf7942 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:02.729317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a93de-ff86-4391-85f5-928e4adf7942 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:02.729798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7a93de-ff86-4391-85f5-928e4adf7942 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:02.732426 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a93de-ff86-4391-85f5-928e4adf7942 None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:02.734085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7a93de-ff86-4391-85f5-928e4adf7942 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:02.736499 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 138/265] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:02 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:02.742876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a93de-ff86-4391-85f5-928e4adf7942 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:02.823052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:02.828879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 128/266] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 794 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:02.834918 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ed126-1198-4009-943a-dd6898924623 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:02.865153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:02.939528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7a93de-ff86-4391-85f5-928e4adf7942 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:03.012140 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f4623-e8c6-4147-9e7f-3d114d363b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:03.012762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f4623-e8c6-4147-9e7f-3d114d363b4a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:03.013040 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f4623-e8c6-4147-9e7f-3d114d363b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:03.013378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f4623-e8c6-4147-9e7f-3d114d363b4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:03.014364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f4623-e8c6-4147-9e7f-3d114d363b4a None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=887965a382184db9958ad175261f03a1, user.id=136255de56114dec8740f51546436962) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:03.014678 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f4623-e8c6-4147-9e7f-3d114d363b4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:03.016148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336f4623-e8c6-4147-9e7f-3d114d363b4a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:03.029239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 139/267] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:02 2022] GET /identity/v3/role_assignments?user.id=136255de56114dec8740f51546436962&scope.project.id=887965a382184db9958ad175261f03a1&role.id=07626899472b4e32adc0c71ce796a25d => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:03.035728 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ed126-1198-4009-943a-dd6898924623 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:03.069684 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67a886-c0a3-4dee-ad99-f2a9d68387db None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:03.069996 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67a886-c0a3-4dee-ad99-f2a9d68387db None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:03.070222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67a886-c0a3-4dee-ad99-f2a9d68387db None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:03.071808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 140/268] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:44:03.077982 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf9ce0-b5bd-4f36-96b4-d60e4d6e3c50 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:03.078277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf9ce0-b5bd-4f36-96b4-d60e4d6e3c50 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:03.078528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf9ce0-b5bd-4f36-96b4-d60e4d6e3c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:03.117519 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ababec-3706-4edf-8e37-7f8a012bc790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:03.118184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ababec-3706-4edf-8e37-7f8a012bc790 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:03.118461 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ababec-3706-4edf-8e37-7f8a012bc790 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:03.118728 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ababec-3706-4edf-8e37-7f8a012bc790 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:03.120627 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ababec-3706-4edf-8e37-7f8a012bc790 None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:03.122168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ababec-3706-4edf-8e37-7f8a012bc790 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:03.123864 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-aadf9ce0-b5bd-4f36-96b4-d60e4d6e3c50 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:03.125203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-aadf9ce0-b5bd-4f36-96b4-d60e4d6e3c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:03.128257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 129/269] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:02 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:03.153985 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ababec-3706-4edf-8e37-7f8a012bc790 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:03.252107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:03.264016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 141/270] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:03.300211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cac55-bb53-459d-9492-72a2fc1b5f1c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:03.301190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cac55-bb53-459d-9492-72a2fc1b5f1c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:03.301681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434cac55-bb53-459d-9492-72a2fc1b5f1c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:03.303107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 142/271] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:44:03.306627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2f5e0-52cf-4e3a-be2b-546d92293fd4 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:03.307039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2f5e0-52cf-4e3a-be2b-546d92293fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:03.307393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b2f5e0-52cf-4e3a-be2b-546d92293fd4 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:03.309876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 143/272] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:03.311086 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee713b-bd4c-4b5b-92c2-e4ecb42e344c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:03.311374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee713b-bd4c-4b5b-92c2-e4ecb42e344c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:03.311680 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee713b-bd4c-4b5b-92c2-e4ecb42e344c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:03.324468 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ababec-3706-4edf-8e37-7f8a012bc790 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:03.341437 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-44ee713b-bd4c-4b5b-92c2-e4ecb42e344c None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:03.343407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-44ee713b-bd4c-4b5b-92c2-e4ecb42e344c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:03.366916 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39677d74-0dc4-4fe3-a154-d82dccff9d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:03.367530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39677d74-0dc4-4fe3-a154-d82dccff9d3b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:03.367872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39677d74-0dc4-4fe3-a154-d82dccff9d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:03.368148 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39677d74-0dc4-4fe3-a154-d82dccff9d3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:03.369097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39677d74-0dc4-4fe3-a154-d82dccff9d3b None admin] RBAC: Adding query filter params (role.id=30bb990dfe5c45fbbfed21c08c7859b4, scope.project.id=887965a382184db9958ad175261f03a1, user.id=136255de56114dec8740f51546436962) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:03.369217 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39677d74-0dc4-4fe3-a154-d82dccff9d3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:03.370725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39677d74-0dc4-4fe3-a154-d82dccff9d3b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:03.377466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 130/273] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:03 2022] GET /identity/v3/role_assignments?user.id=136255de56114dec8740f51546436962&scope.project.id=887965a382184db9958ad175261f03a1&role.id=30bb990dfe5c45fbbfed21c08c7859b4 => generated 608 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:03.379314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e411e-041d-4c46-a74e-69b28a171073 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:03.383719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e411e-041d-4c46-a74e-69b28a171073 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:03.383719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e411e-041d-4c46-a74e-69b28a171073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:03.408403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-168e411e-041d-4c46-a74e-69b28a171073 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:03.410715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-168e411e-041d-4c46-a74e-69b28a171073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:03.452991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:03.477107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 144/274] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:03.523520 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 131/275] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:03.592935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee713b-bd4c-4b5b-92c2-e4ecb42e344c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:03.716897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee713b-bd4c-4b5b-92c2-e4ecb42e344c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:03.752064 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3c48c-0ca4-489b-96e0-1c70badff2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:03.752745 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3c48c-0ca4-489b-96e0-1c70badff2fa None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:03.753030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3c48c-0ca4-489b-96e0-1c70badff2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:03.753299 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd3c48c-0ca4-489b-96e0-1c70badff2fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:03.754963 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3c48c-0ca4-489b-96e0-1c70badff2fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:03.756443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd3c48c-0ca4-489b-96e0-1c70badff2fa None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:03.758379 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 145/276] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:03.763486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e411e-041d-4c46-a74e-69b28a171073 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:03.827915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:03.837737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3c48c-0ca4-489b-96e0-1c70badff2fa None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:03.882350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e411e-041d-4c46-a74e-69b28a171073 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:03.908060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:03.922525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2154eed-22ca-4465-abfb-da113786ad72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:03.923147 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2154eed-22ca-4465-abfb-da113786ad72 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:03.923424 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2154eed-22ca-4465-abfb-da113786ad72 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:03.923813 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2154eed-22ca-4465-abfb-da113786ad72 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:03.924484 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2154eed-22ca-4465-abfb-da113786ad72 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:03.926675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2154eed-22ca-4465-abfb-da113786ad72 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:03.951458 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd3c48c-0ca4-489b-96e0-1c70badff2fa None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:03.986726 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e37bc4-d9e6-4bcc-85d0-f4b6af6028db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:03.987385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e37bc4-d9e6-4bcc-85d0-f4b6af6028db None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:03.987695 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e37bc4-d9e6-4bcc-85d0-f4b6af6028db None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:03.988626 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e37bc4-d9e6-4bcc-85d0-f4b6af6028db None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:03.998640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e37bc4-d9e6-4bcc-85d0-f4b6af6028db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-80e37bc4-d9e6-4bcc-85d0-f4b6af6028db None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:04.003698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:04.005402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:04.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:04.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:04.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:04.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:04.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:04.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:04.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:04.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:04.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:04.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:04.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:04.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:04.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 146/277] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:04.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e37bc4-d9e6-4bcc-85d0-f4b6af6028db None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:04.041990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 132/278] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:44:03 2022] POST /identity/v3/users => generated 293 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:44:04.043033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2154eed-22ca-4465-abfb-da113786ad72 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:04.132858 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e37bc4-d9e6-4bcc-85d0-f4b6af6028db None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:04.161043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:04.162773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2154eed-22ca-4465-abfb-da113786ad72 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:04.181898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3952fe2-41d0-4a1f-b0a0-0929575c1fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:04.182851 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3952fe2-41d0-4a1f-b0a0-0929575c1fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:04.183710 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3952fe2-41d0-4a1f-b0a0-0929575c1fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:04.184768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:04.185483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3952fe2-41d0-4a1f-b0a0-0929575c1fa8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:04.201242 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3952fe2-41d0-4a1f-b0a0-0929575c1fa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:04.203757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da9acf-1b6c-422e-8fe1-fb6c75a8c047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:04.204390 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da9acf-1b6c-422e-8fe1-fb6c75a8c047 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:04.204765 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da9acf-1b6c-422e-8fe1-fb6c75a8c047 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:04.205438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da9acf-1b6c-422e-8fe1-fb6c75a8c047 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:04.206201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da9acf-1b6c-422e-8fe1-fb6c75a8c047 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:04.206577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da9acf-1b6c-422e-8fe1-fb6c75a8c047 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b3952fe2-41d0-4a1f-b0a0-0929575c1fa8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:04.208805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:04.209662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:04.210416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:04.210416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:04.210416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:04.210416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:04.210416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:04.210416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:04.210416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:04.210416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:04.210416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:04.210416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:04.210416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:44:04.210416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:04.210982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da9acf-1b6c-422e-8fe1-fb6c75a8c047 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:04.213391 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 147/279] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:04.219514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3952fe2-41d0-4a1f-b0a0-0929575c1fa8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:04.226489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 133/280] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:04.232785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da9acf-1b6c-422e-8fe1-fb6c75a8c047 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:04.357247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3952fe2-41d0-4a1f-b0a0-0929575c1fa8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:04.369054 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da9acf-1b6c-422e-8fe1-fb6c75a8c047 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:04.399290 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a014f60-b0e1-4544-8ede-87f8fc311aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:04.399878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a014f60-b0e1-4544-8ede-87f8fc311aea None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:04.400157 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a014f60-b0e1-4544-8ede-87f8fc311aea None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:04.400428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a014f60-b0e1-4544-8ede-87f8fc311aea None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:04.401087 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a014f60-b0e1-4544-8ede-87f8fc311aea None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:04.401386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a014f60-b0e1-4544-8ede-87f8fc311aea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:04.402968 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a014f60-b0e1-4544-8ede-87f8fc311aea None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:04.414505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 148/281] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:04.421227 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee74495-99e9-4196-bc31-ee0ad24d6290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:04.421887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee74495-99e9-4196-bc31-ee0ad24d6290 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:04.422168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee74495-99e9-4196-bc31-ee0ad24d6290 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:04.422439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee74495-99e9-4196-bc31-ee0ad24d6290 None admin] PATH_INFO: `/v3/users/c54bb29c001640a3beebd69641326465` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:04.423913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a014f60-b0e1-4544-8ede-87f8fc311aea None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:04.439017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee74495-99e9-4196-bc31-ee0ad24d6290 None admin] RBAC: Authorizing `identity:get_user(user_id=c54bb29c001640a3beebd69641326465)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:04.440583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee74495-99e9-4196-bc31-ee0ad24d6290 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:04.443539 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 134/282] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:04 2022] GET /identity/v3/users/c54bb29c001640a3beebd69641326465 => generated 293 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:04.449166 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee74495-99e9-4196-bc31-ee0ad24d6290 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:04.479304 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:04.496653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:04.563171 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a014f60-b0e1-4544-8ede-87f8fc311aea None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:04.568363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee74495-99e9-4196-bc31-ee0ad24d6290 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:04.607209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd32fa8-d977-40cc-9217-e0fc5da2ee14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:04.607968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd32fa8-d977-40cc-9217-e0fc5da2ee14 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:04.608462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd32fa8-d977-40cc-9217-e0fc5da2ee14 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:04.608870 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd32fa8-d977-40cc-9217-e0fc5da2ee14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:04.609643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd32fa8-d977-40cc-9217-e0fc5da2ee14 None admin] RBAC: Adding query filter params (role.id=1c983c7e32e44c61be3996c28399e617, user.id=c54bb29c001640a3beebd69641326465, scope.system=all) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:04.609954 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd32fa8-d977-40cc-9217-e0fc5da2ee14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:04.610140 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77f2c3-6a32-4e79-a08e-2c1b319b4186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:04.610927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77f2c3-6a32-4e79-a08e-2c1b319b4186 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:04.611349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77f2c3-6a32-4e79-a08e-2c1b319b4186 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:04.611468 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd32fa8-d977-40cc-9217-e0fc5da2ee14 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:04.611971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c77f2c3-6a32-4e79-a08e-2c1b319b4186 None admin] PATH_INFO: `/v3/users/74ea797808eb402eb58427ea8e599a00` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:04.614459 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c77f2c3-6a32-4e79-a08e-2c1b319b4186 None admin] RBAC: Authorizing `identity:get_user(user_id=74ea797808eb402eb58427ea8e599a00)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:04.616787 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c77f2c3-6a32-4e79-a08e-2c1b319b4186 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:04.617890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 135/283] 10.222.0.67 () {56 vars in 1308 bytes} [Mon Sep 26 17:44:04 2022] GET /identity/v3/role_assignments?user.id=c54bb29c001640a3beebd69641326465&scope.system=all&role.id=1c983c7e32e44c61be3996c28399e617 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:04.618738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 149/284] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:04 2022] GET /identity/v3/users/74ea797808eb402eb58427ea8e599a00 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:04.624527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd32fa8-d977-40cc-9217-e0fc5da2ee14 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:04.748350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd32fa8-d977-40cc-9217-e0fc5da2ee14 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:04.750040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:04.797071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e518e9c-7270-43ef-800a-c3a4623941a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:04.797724 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e518e9c-7270-43ef-800a-c3a4623941a4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:04.798071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e518e9c-7270-43ef-800a-c3a4623941a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:04.798348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e518e9c-7270-43ef-800a-c3a4623941a4 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:04.799608 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e518e9c-7270-43ef-800a-c3a4623941a4 None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:04.801194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e518e9c-7270-43ef-800a-c3a4623941a4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:04.803196 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 136/285] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:04 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:04.809474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77f2c3-6a32-4e79-a08e-2c1b319b4186 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:04.933616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c77f2c3-6a32-4e79-a08e-2c1b319b4186 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:04.945793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:04.957549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98dcf46-312e-43b8-a295-9834190cceb0 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:04.958057 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98dcf46-312e-43b8-a295-9834190cceb0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:04.958275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98dcf46-312e-43b8-a295-9834190cceb0 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:04.960023 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 137/286] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:04.965530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2401dd-d785-45d8-b62c-0ea48f1fe5e6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:04.965809 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2401dd-d785-45d8-b62c-0ea48f1fe5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:04.966081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2401dd-d785-45d8-b62c-0ea48f1fe5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:04.993082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9936041-d031-4cb6-a64f-fbe1fccc785e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:04.993760 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9936041-d031-4cb6-a64f-fbe1fccc785e None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:04.994035 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9936041-d031-4cb6-a64f-fbe1fccc785e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:04.994308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9936041-d031-4cb6-a64f-fbe1fccc785e None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1/users/74ea797808eb402eb58427ea8e599a00/roles/6a413519e66945239f17395ca12293c2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:04.996357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8d2401dd-d785-45d8-b62c-0ea48f1fe5e6 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:04.997538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8d2401dd-d785-45d8-b62c-0ea48f1fe5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:05.000254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9936041-d031-4cb6-a64f-fbe1fccc785e None admin] RBAC: Authorizing `identity:create_grant(project_id=887965a382184db9958ad175261f03a1, user_id=74ea797808eb402eb58427ea8e599a00, role_id=6a413519e66945239f17395ca12293c2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:05.001687 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9936041-d031-4cb6-a64f-fbe1fccc785e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:05.078413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 138/287] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:05.093567 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 150/288] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:44:04 2022] PUT /identity/v3/projects/887965a382184db9958ad175261f03a1/users/74ea797808eb402eb58427ea8e599a00/roles/6a413519e66945239f17395ca12293c2 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:05.365077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2401dd-d785-45d8-b62c-0ea48f1fe5e6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:05.506777 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:05.511920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2401dd-d785-45d8-b62c-0ea48f1fe5e6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:05.560481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfa639-f0b3-4535-b1dd-b1207d6f217a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:05.561121 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfa639-f0b3-4535-b1dd-b1207d6f217a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:05.561400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfa639-f0b3-4535-b1dd-b1207d6f217a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:05.561672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfa639-f0b3-4535-b1dd-b1207d6f217a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:05.577789 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfa639-f0b3-4535-b1dd-b1207d6f217a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a3bfa639-f0b3-4535-b1dd-b1207d6f217a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:05.583687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:05.584540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:05.585324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:05.585324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:05.585324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:05.585324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:05.585324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:05.585324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:05.585324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:05.585324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:05.585324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:05.585324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:05.585324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:05.585324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:05.590087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 139/289] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:05.592076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9936041-d031-4cb6-a64f-fbe1fccc785e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:05.821582 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9936041-d031-4cb6-a64f-fbe1fccc785e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:05.869645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29d605-8bab-40ba-b660-7719183e46e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:05.870259 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29d605-8bab-40ba-b660-7719183e46e0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:05.870545 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29d605-8bab-40ba-b660-7719183e46e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:05.870822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29d605-8bab-40ba-b660-7719183e46e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:05.871515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29d605-8bab-40ba-b660-7719183e46e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:05.871844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29d605-8bab-40ba-b660-7719183e46e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:05.873218 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29d605-8bab-40ba-b660-7719183e46e0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:05.886094 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 151/290] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:05.892108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfa639-f0b3-4535-b1dd-b1207d6f217a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:06.054922 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfa639-f0b3-4535-b1dd-b1207d6f217a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:06.112621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653242b-017d-45ef-9e7e-2376a88033bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:06.113895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653242b-017d-45ef-9e7e-2376a88033bb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:06.114175 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653242b-017d-45ef-9e7e-2376a88033bb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:06.114445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653242b-017d-45ef-9e7e-2376a88033bb None admin] PATH_INFO: `/v3/users/4b32ea2da2eb41e0bd5d9a8cd9d690bb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:06.120329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653242b-017d-45ef-9e7e-2376a88033bb None admin] RBAC: Authorizing `identity:get_user(user_id=4b32ea2da2eb41e0bd5d9a8cd9d690bb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:06.121784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653242b-017d-45ef-9e7e-2376a88033bb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:06.123365 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 140/291] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:05 2022] GET /identity/v3/users/4b32ea2da2eb41e0bd5d9a8cd9d690bb => generated 297 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:06.128884 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29d605-8bab-40ba-b660-7719183e46e0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:06.181067 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:06.274762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29d605-8bab-40ba-b660-7719183e46e0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:06.328806 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d0afb-0f91-40c6-b912-99a69d73940c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:06.329428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d0afb-0f91-40c6-b912-99a69d73940c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:06.329715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d0afb-0f91-40c6-b912-99a69d73940c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:06.329991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d0afb-0f91-40c6-b912-99a69d73940c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:06.431128 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d0afb-0f91-40c6-b912-99a69d73940c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:06.432631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d0afb-0f91-40c6-b912-99a69d73940c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:06.474366 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:06.569230 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 152/292] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:06.574375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653242b-017d-45ef-9e7e-2376a88033bb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:06.610245 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:06.710272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653242b-017d-45ef-9e7e-2376a88033bb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:06.772388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3718bb-9b0f-40c6-8853-55b794e0d101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:06.773041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3718bb-9b0f-40c6-8853-55b794e0d101 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:06.773324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3718bb-9b0f-40c6-8853-55b794e0d101 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:06.773597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3718bb-9b0f-40c6-8853-55b794e0d101 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:06.774794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3718bb-9b0f-40c6-8853-55b794e0d101 None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:06.776280 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3718bb-9b0f-40c6-8853-55b794e0d101 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:06.778201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 141/293] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:06 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:06.784198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d0afb-0f91-40c6-b912-99a69d73940c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:06.952043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d0afb-0f91-40c6-b912-99a69d73940c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:07.020052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:07.025019 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643c1ff-40bc-4693-a18c-c5e672d5cc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:07.025630 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643c1ff-40bc-4693-a18c-c5e672d5cc43 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:07.025908 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643c1ff-40bc-4693-a18c-c5e672d5cc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:07.026178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643c1ff-40bc-4693-a18c-c5e672d5cc43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:07.026891 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643c1ff-40bc-4693-a18c-c5e672d5cc43 None admin] RBAC: Adding query filter params (role.id=1c983c7e32e44c61be3996c28399e617, scope.project.id=887965a382184db9958ad175261f03a1, user.id=4b32ea2da2eb41e0bd5d9a8cd9d690bb) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:07.027193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643c1ff-40bc-4693-a18c-c5e672d5cc43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:07.028650 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643c1ff-40bc-4693-a18c-c5e672d5cc43 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:07.040321 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 153/294] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:06 2022] GET /identity/v3/role_assignments?user.id=4b32ea2da2eb41e0bd5d9a8cd9d690bb&scope.project.id=887965a382184db9958ad175261f03a1&role.id=1c983c7e32e44c61be3996c28399e617 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:07.182754 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5786b-c27e-488f-bb59-a61a2add9ede None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:07.183833 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5786b-c27e-488f-bb59-a61a2add9ede None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:07.184357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5786b-c27e-488f-bb59-a61a2add9ede None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:07.186425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 142/295] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:07.197851 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf4a2e-1b73-4f54-90e9-405ba9c3b10d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:07.198159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf4a2e-1b73-4f54-90e9-405ba9c3b10d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:07.198410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf4a2e-1b73-4f54-90e9-405ba9c3b10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:07.199866 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234e9e0-3270-4cea-9a5b-14b1b65c6e29 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:07.200360 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234e9e0-3270-4cea-9a5b-14b1b65c6e29 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:07.200829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f234e9e0-3270-4cea-9a5b-14b1b65c6e29 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:07.202426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 143/296] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:44:07.208775 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03b068-f6ff-4a71-8052-0d50843f1be0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:07.209344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03b068-f6ff-4a71-8052-0d50843f1be0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:07.209851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03b068-f6ff-4a71-8052-0d50843f1be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:07.232285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c6bf4a2e-1b73-4f54-90e9-405ba9c3b10d None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:07.233496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c6bf4a2e-1b73-4f54-90e9-405ba9c3b10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:07.251139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fd03b068-f6ff-4a71-8052-0d50843f1be0 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:07.252320 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fd03b068-f6ff-4a71-8052-0d50843f1be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:07.344664 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 144/297] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:07.346255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21040ca1-2522-4810-b69c-36d21e882d95 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:07.346759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21040ca1-2522-4810-b69c-36d21e882d95 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:07.347287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21040ca1-2522-4810-b69c-36d21e882d95 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:07.348255 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 154/298] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:07.350540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 145/299] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:07.355916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb1d7c-4db6-447d-af31-643639a4ebec None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:07.356444 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb1d7c-4db6-447d-af31-643639a4ebec None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:07.356911 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdb1d7c-4db6-447d-af31-643639a4ebec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:07.376168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8cdb1d7c-4db6-447d-af31-643639a4ebec None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:07.377702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8cdb1d7c-4db6-447d-af31-643639a4ebec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:07.450518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 155/300] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:07.640457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21040ca1-2522-4810-b69c-36d21e882d95 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:07.655508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb1d7c-4db6-447d-af31-643639a4ebec None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:07.703951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:07.743430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:07.791145 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21040ca1-2522-4810-b69c-36d21e882d95 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:07.793706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdb1d7c-4db6-447d-af31-643639a4ebec None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:07.831063 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27dd67f-8a8d-4f09-966a-404d308faea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:07.832315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27dd67f-8a8d-4f09-966a-404d308faea0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:07.832599 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27dd67f-8a8d-4f09-966a-404d308faea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:07.832877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27dd67f-8a8d-4f09-966a-404d308faea0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:07.834173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ac299-69bc-4744-929f-1dc4638d2389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:07.834815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ac299-69bc-4744-929f-1dc4638d2389 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:07.835093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ac299-69bc-4744-929f-1dc4638d2389 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:07.835361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680ac299-69bc-4744-929f-1dc4638d2389 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:07.836713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ac299-69bc-4744-929f-1dc4638d2389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:07.838132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680ac299-69bc-4744-929f-1dc4638d2389 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:07.839903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 156/301] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:07.840462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ac299-69bc-4744-929f-1dc4638d2389 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:07.850694 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27dd67f-8a8d-4f09-966a-404d308faea0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e27dd67f-8a8d-4f09-966a-404d308faea0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:07.857238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:07.858099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:07.859194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:07.859194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:07.859194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:07.859194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:07.859194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:07.859194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:07.859194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:07.859194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:07.859194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:07.859194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:07.859194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:44:07.859194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:07.859194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 146/302] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:07.863782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27dd67f-8a8d-4f09-966a-404d308faea0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:07.947937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680ac299-69bc-4744-929f-1dc4638d2389 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:07.952980 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:07.968602 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27dd67f-8a8d-4f09-966a-404d308faea0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:07.971274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:07.987636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589819a-eb71-4a37-a1b0-f16b7a811100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:07.988420 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589819a-eb71-4a37-a1b0-f16b7a811100 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:07.988697 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589819a-eb71-4a37-a1b0-f16b7a811100 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:07.988972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2589819a-eb71-4a37-a1b0-f16b7a811100 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:07.999982 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2589819a-eb71-4a37-a1b0-f16b7a811100 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:08.004346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650c5ab-751b-4713-ba3d-c23eaafc02ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:08.004958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650c5ab-751b-4713-ba3d-c23eaafc02ad None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2589819a-eb71-4a37-a1b0-f16b7a811100 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:08.005116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:08.005973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:08.006748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:08.006748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:08.006748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:08.006748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:08.006748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:08.006748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:08.006748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:08.006748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:08.006748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:08.006748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:08.006748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:08.006748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:08.007333 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650c5ab-751b-4713-ba3d-c23eaafc02ad None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:08.007333 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650c5ab-751b-4713-ba3d-c23eaafc02ad None admin] PATH_INFO: `/v3/groups` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:08.007333 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650c5ab-751b-4713-ba3d-c23eaafc02ad None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:08.007504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 157/303] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:08.007504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589819a-eb71-4a37-a1b0-f16b7a811100 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:08.007618 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650c5ab-751b-4713-ba3d-c23eaafc02ad None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:08.057298 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 147/304] 10.222.0.67 () {60 vars in 1134 bytes} [Mon Sep 26 17:44:07 2022] POST /identity/v3/groups => generated 229 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:44:08.058526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59879cf-eb83-479e-9ea5-c7a40bc93298 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:08.058811 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59879cf-eb83-479e-9ea5-c7a40bc93298 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:08.059085 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59879cf-eb83-479e-9ea5-c7a40bc93298 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:08.060469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 148/305] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:08.061257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59879cf-eb83-479e-9ea5-c7a40bc93298 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:08.119900 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2589819a-eb71-4a37-a1b0-f16b7a811100 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:08.159614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a276e-6e22-48b5-a923-a1fdd14de325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:08.160507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a276e-6e22-48b5-a923-a1fdd14de325 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:08.160793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a276e-6e22-48b5-a923-a1fdd14de325 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:08.161069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a276e-6e22-48b5-a923-a1fdd14de325 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:08.161746 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a276e-6e22-48b5-a923-a1fdd14de325 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:08.162050 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a276e-6e22-48b5-a923-a1fdd14de325 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:08.163450 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a276e-6e22-48b5-a923-a1fdd14de325 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:08.172452 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 158/306] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:08.173798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33340246-32dc-4ba9-b335-68b2683ab090 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:08.174093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33340246-32dc-4ba9-b335-68b2683ab090 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:08.174370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33340246-32dc-4ba9-b335-68b2683ab090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:08.189805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59879cf-eb83-479e-9ea5-c7a40bc93298 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:08.194811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-33340246-32dc-4ba9-b335-68b2683ab090 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:08.196353 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-33340246-32dc-4ba9-b335-68b2683ab090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:08.228673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150d84d-dee7-41ed-a1cb-1d078080c984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:08.229213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150d84d-dee7-41ed-a1cb-1d078080c984 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:08.229497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150d84d-dee7-41ed-a1cb-1d078080c984 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:08.229777 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6150d84d-dee7-41ed-a1cb-1d078080c984 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:08.230452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150d84d-dee7-41ed-a1cb-1d078080c984 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:08.230763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150d84d-dee7-41ed-a1cb-1d078080c984 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:08.232209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6150d84d-dee7-41ed-a1cb-1d078080c984 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:08.244517 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 149/307] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:08.245622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150d84d-dee7-41ed-a1cb-1d078080c984 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:08.249396 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:08.258458 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:08.279551 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 159/308] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:08.280512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33340246-32dc-4ba9-b335-68b2683ab090 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:08.363997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6150d84d-dee7-41ed-a1cb-1d078080c984 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:08.406530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605742d-ed58-4625-adec-d4aad29a88d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:08.407721 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605742d-ed58-4625-adec-d4aad29a88d7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:08.408176 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605742d-ed58-4625-adec-d4aad29a88d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:08.408647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9605742d-ed58-4625-adec-d4aad29a88d7 None admin] PATH_INFO: `/v3/users/a9f619bf1e444fbe8455ce5171a8bb30` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:08.409175 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33340246-32dc-4ba9-b335-68b2683ab090 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:08.426365 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605742d-ed58-4625-adec-d4aad29a88d7 None admin] RBAC: Authorizing `identity:get_user(user_id=a9f619bf1e444fbe8455ce5171a8bb30)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:08.428914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9605742d-ed58-4625-adec-d4aad29a88d7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:08.431384 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 150/309] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:08 2022] GET /identity/v3/users/a9f619bf1e444fbe8455ce5171a8bb30 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:08.437837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605742d-ed58-4625-adec-d4aad29a88d7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:08.454765 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387add1-f82f-46d6-8b4c-62d7779674cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:08.455928 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387add1-f82f-46d6-8b4c-62d7779674cb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:08.456285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387add1-f82f-46d6-8b4c-62d7779674cb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:08.457043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f387add1-f82f-46d6-8b4c-62d7779674cb None admin] PATH_INFO: `/v3/users/c54bb29c001640a3beebd69641326465` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:08.458338 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387add1-f82f-46d6-8b4c-62d7779674cb None admin] RBAC: Authorizing `identity:get_user(user_id=c54bb29c001640a3beebd69641326465)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:08.460325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f387add1-f82f-46d6-8b4c-62d7779674cb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:08.462396 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 160/310] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:08 2022] GET /identity/v3/users/c54bb29c001640a3beebd69641326465 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:08.467427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387add1-f82f-46d6-8b4c-62d7779674cb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:08.520648 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:08.527274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:08.563995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9605742d-ed58-4625-adec-d4aad29a88d7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:08.588533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f387add1-f82f-46d6-8b4c-62d7779674cb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:08.600654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae154a-c878-4926-bdc8-e545c141010e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:08.601266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae154a-c878-4926-bdc8-e545c141010e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:08.601551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae154a-c878-4926-bdc8-e545c141010e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:08.601827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae154a-c878-4926-bdc8-e545c141010e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:08.602576 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae154a-c878-4926-bdc8-e545c141010e None admin] RBAC: Adding query filter params (role.id=6a413519e66945239f17395ca12293c2, user.id=a9f619bf1e444fbe8455ce5171a8bb30, scope.system=all) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:08.602908 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae154a-c878-4926-bdc8-e545c141010e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:08.604348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae154a-c878-4926-bdc8-e545c141010e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:08.619859 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 151/311] 10.222.0.67 () {56 vars in 1308 bytes} [Mon Sep 26 17:44:08 2022] GET /identity/v3/role_assignments?user.id=a9f619bf1e444fbe8455ce5171a8bb30&scope.system=all&role.id=6a413519e66945239f17395ca12293c2 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:08.619859 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae154a-c878-4926-bdc8-e545c141010e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:08.640091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad151e-1d75-4d37-9773-2554ae52be08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:08.640781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad151e-1d75-4d37-9773-2554ae52be08 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:08.641064 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad151e-1d75-4d37-9773-2554ae52be08 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:08.641341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad151e-1d75-4d37-9773-2554ae52be08 None admin] PATH_INFO: `/v3/system/users/c54bb29c001640a3beebd69641326465/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:08.643055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad151e-1d75-4d37-9773-2554ae52be08 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c54bb29c001640a3beebd69641326465, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:08.645683 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad151e-1d75-4d37-9773-2554ae52be08 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:08.711967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 161/312] 10.222.0.67 () {58 vars in 1334 bytes} [Mon Sep 26 17:44:08 2022] PUT /identity/v3/system/users/c54bb29c001640a3beebd69641326465/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 17:44:08.751446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae154a-c878-4926-bdc8-e545c141010e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:08.789973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:08.797742 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57ad46-8944-4458-ab8a-bb978ed531e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:08.798409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57ad46-8944-4458-ab8a-bb978ed531e6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:08.798691 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57ad46-8944-4458-ab8a-bb978ed531e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:08.799020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e57ad46-8944-4458-ab8a-bb978ed531e6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:08.811072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e57ad46-8944-4458-ab8a-bb978ed531e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5e57ad46-8944-4458-ab8a-bb978ed531e6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:08.816471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:08.817583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:08.818383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:08.818383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:08.818383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:08.818383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:08.818383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:08.818383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:08.818383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:08.818383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:08.818383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:08.818383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:08.818383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:44:08.818383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:08.818383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 152/313] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:08.824467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad151e-1d75-4d37-9773-2554ae52be08 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:08.946697 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad151e-1d75-4d37-9773-2554ae52be08 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:08.965491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:08.984128 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a526ab1-3673-4036-83e5-2f81f1b49989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:08.984730 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a526ab1-3673-4036-83e5-2f81f1b49989 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:08.985005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a526ab1-3673-4036-83e5-2f81f1b49989 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:08.985275 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a526ab1-3673-4036-83e5-2f81f1b49989 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:08.985937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a526ab1-3673-4036-83e5-2f81f1b49989 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:08.986238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a526ab1-3673-4036-83e5-2f81f1b49989 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:08.987598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a526ab1-3673-4036-83e5-2f81f1b49989 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:08.997709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 162/314] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:09.004003 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57ad46-8944-4458-ab8a-bb978ed531e6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:09.125130 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e57ad46-8944-4458-ab8a-bb978ed531e6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:09.162579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b05df2-45ce-459e-9fd9-3ec9e854f67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:09.163212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b05df2-45ce-459e-9fd9-3ec9e854f67d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:09.163493 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b05df2-45ce-459e-9fd9-3ec9e854f67d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:09.163800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b05df2-45ce-459e-9fd9-3ec9e854f67d None admin] PATH_INFO: `/v3/users/74ea797808eb402eb58427ea8e599a00` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:09.164932 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b05df2-45ce-459e-9fd9-3ec9e854f67d None admin] RBAC: Authorizing `identity:get_user(user_id=74ea797808eb402eb58427ea8e599a00)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:09.166550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b05df2-45ce-459e-9fd9-3ec9e854f67d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:09.168161 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 153/315] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:09 2022] GET /identity/v3/users/74ea797808eb402eb58427ea8e599a00 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:09.173723 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a526ab1-3673-4036-83e5-2f81f1b49989 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:09.278207 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a526ab1-3673-4036-83e5-2f81f1b49989 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:09.314918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194e8f2-e040-4073-ba55-7630103bfa0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:09.315531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194e8f2-e040-4073-ba55-7630103bfa0c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:09.315848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194e8f2-e040-4073-ba55-7630103bfa0c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:09.316700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194e8f2-e040-4073-ba55-7630103bfa0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:09.361902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:09.397954 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194e8f2-e040-4073-ba55-7630103bfa0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:09.400725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b194e8f2-e040-4073-ba55-7630103bfa0c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:09.507384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 163/316] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:09.513322 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b05df2-45ce-459e-9fd9-3ec9e854f67d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:09.585881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:09.630156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b05df2-45ce-459e-9fd9-3ec9e854f67d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:09.667759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11706fe3-86c6-408a-bc3a-71fdb5159e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:09.668603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11706fe3-86c6-408a-bc3a-71fdb5159e55 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:09.669020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11706fe3-86c6-408a-bc3a-71fdb5159e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:09.669431 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11706fe3-86c6-408a-bc3a-71fdb5159e55 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:09.670694 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11706fe3-86c6-408a-bc3a-71fdb5159e55 None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:09.672393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11706fe3-86c6-408a-bc3a-71fdb5159e55 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:09.674461 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 154/317] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:09 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:09.680749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194e8f2-e040-4073-ba55-7630103bfa0c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:09.758650 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:09.792181 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194e8f2-e040-4073-ba55-7630103bfa0c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:09.827867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182cba8-a77c-45e6-be4b-06eda2ad7072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:09.828783 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182cba8-a77c-45e6-be4b-06eda2ad7072 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:09.829213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182cba8-a77c-45e6-be4b-06eda2ad7072 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:09.829961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182cba8-a77c-45e6-be4b-06eda2ad7072 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:09.830833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182cba8-a77c-45e6-be4b-06eda2ad7072 None admin] RBAC: Adding query filter params (role.id=6a413519e66945239f17395ca12293c2, scope.project.id=887965a382184db9958ad175261f03a1, user.id=74ea797808eb402eb58427ea8e599a00) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:09.831383 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182cba8-a77c-45e6-be4b-06eda2ad7072 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:09.833618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182cba8-a77c-45e6-be4b-06eda2ad7072 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:09.840190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 164/318] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:09 2022] GET /identity/v3/role_assignments?user.id=74ea797808eb402eb58427ea8e599a00&scope.project.id=887965a382184db9958ad175261f03a1&role.id=6a413519e66945239f17395ca12293c2 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:10.383009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97e47-7239-4b22-b40a-e5c533ac9bda None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:10.383370 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97e47-7239-4b22-b40a-e5c533ac9bda None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:10.383567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97e47-7239-4b22-b40a-e5c533ac9bda None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:10.384979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 155/319] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:10.390320 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9160297-4d21-443f-9529-12b01fed28c1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:10.390797 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9160297-4d21-443f-9529-12b01fed28c1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:10.391085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9160297-4d21-443f-9529-12b01fed28c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:10.410872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c9160297-4d21-443f-9529-12b01fed28c1 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:10.412088 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c9160297-4d21-443f-9529-12b01fed28c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:10.483692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 165/320] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:10.754945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea97e47-7239-4b22-b40a-e5c533ac9bda None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:10.837227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b83718-f8b7-47e6-ad17-230e219eb82a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:10.837516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b83718-f8b7-47e6-ad17-230e219eb82a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:10.837795 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b83718-f8b7-47e6-ad17-230e219eb82a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:10.839259 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 166/321] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:44:10.843825 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62501b2f-cd2a-4221-9ccc-6bd01fea7ff4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:10.844102 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62501b2f-cd2a-4221-9ccc-6bd01fea7ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:10.844381 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62501b2f-cd2a-4221-9ccc-6bd01fea7ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:10.861388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea97e47-7239-4b22-b40a-e5c533ac9bda None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:10.864167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-62501b2f-cd2a-4221-9ccc-6bd01fea7ff4 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:10.865353 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-62501b2f-cd2a-4221-9ccc-6bd01fea7ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:10.890233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:10.900339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a17e2-5036-499e-be23-8a6842b7ed34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:10.900970 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a17e2-5036-499e-be23-8a6842b7ed34 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:10.901243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a17e2-5036-499e-be23-8a6842b7ed34 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:10.901510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a17e2-5036-499e-be23-8a6842b7ed34 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:10.904281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a17e2-5036-499e-be23-8a6842b7ed34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:10.904454 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a17e2-5036-499e-be23-8a6842b7ed34 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:10.908116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 156/322] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:10.911886 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a17e2-5036-499e-be23-8a6842b7ed34 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:10.930139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:10.938431 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 167/323] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:11.027126 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a17e2-5036-499e-be23-8a6842b7ed34 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:11.062304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441a328-702a-46c1-8f83-5a7861baa8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:11.062944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441a328-702a-46c1-8f83-5a7861baa8bb None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:11.063220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441a328-702a-46c1-8f83-5a7861baa8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:11.063487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8441a328-702a-46c1-8f83-5a7861baa8bb None admin] PATH_INFO: `/v3/groups` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:11.064133 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441a328-702a-46c1-8f83-5a7861baa8bb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:11.065566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8441a328-702a-46c1-8f83-5a7861baa8bb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:11.071237 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66dbd2-45ee-4404-b6d5-d83008177557 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:11.071528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66dbd2-45ee-4404-b6d5-d83008177557 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:11.071830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b66dbd2-45ee-4404-b6d5-d83008177557 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:11.073174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 168/324] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:44:11.078183 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ce3a8-2877-4a6d-8a54-6ca4350b5f5e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:11.078524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ce3a8-2877-4a6d-8a54-6ca4350b5f5e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:11.082059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94ce3a8-2877-4a6d-8a54-6ca4350b5f5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:11.097751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a94ce3a8-2877-4a6d-8a54-6ca4350b5f5e None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:11.099082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a94ce3a8-2877-4a6d-8a54-6ca4350b5f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:11.143470 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 157/325] 10.222.0.67 () {60 vars in 1134 bytes} [Mon Sep 26 17:44:10 2022] POST /identity/v3/groups => generated 226 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:44:11.173210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 169/326] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:11.219987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441a328-702a-46c1-8f83-5a7861baa8bb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:11.280686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:11.332716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8441a328-702a-46c1-8f83-5a7861baa8bb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:11.369596 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a4aa2-47ff-4491-9f4e-12032311a96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:11.370235 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a4aa2-47ff-4491-9f4e-12032311a96d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:11.370514 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a4aa2-47ff-4491-9f4e-12032311a96d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:11.370782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a4aa2-47ff-4491-9f4e-12032311a96d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:11.382161 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a4aa2-47ff-4491-9f4e-12032311a96d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e95a4aa2-47ff-4491-9f4e-12032311a96d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:11.387552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:11.388481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:11.389731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:11.389731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:11.389731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:11.389731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:11.389731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:11.389731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:11.389731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:11.389731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:11.389731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:11.389731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:11.389731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:44:11.389731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:11.389731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 158/327] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:11.394937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ce3a8-2877-4a6d-8a54-6ca4350b5f5e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:11.486413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a4aa2-47ff-4491-9f4e-12032311a96d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:11.508618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94ce3a8-2877-4a6d-8a54-6ca4350b5f5e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:11.540522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:11.549903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4630a6-0352-444d-883d-7012c1ecf4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:11.551277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4630a6-0352-444d-883d-7012c1ecf4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:11.551754 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4630a6-0352-444d-883d-7012c1ecf4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:11.552393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4630a6-0352-444d-883d-7012c1ecf4c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:11.552715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4630a6-0352-444d-883d-7012c1ecf4c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:11.553018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4630a6-0352-444d-883d-7012c1ecf4c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:11.554385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4630a6-0352-444d-883d-7012c1ecf4c6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:11.564164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 170/328] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:11.570033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4630a6-0352-444d-883d-7012c1ecf4c6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:11.615338 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a4aa2-47ff-4491-9f4e-12032311a96d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:11.634737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:11.654924 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999248f-0a12-4a81-8ba3-c26844e4833a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:11.655559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999248f-0a12-4a81-8ba3-c26844e4833a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:11.655867 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999248f-0a12-4a81-8ba3-c26844e4833a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:11.656162 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999248f-0a12-4a81-8ba3-c26844e4833a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:11.672507 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999248f-0a12-4a81-8ba3-c26844e4833a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a999248f-0a12-4a81-8ba3-c26844e4833a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:11.680449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:11.681841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:11.683103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:11.683103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:11.683103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:11.683103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:11.683103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:11.683103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:11.683103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:11.683103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:11.683103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:11.683103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:11.683103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:11.683103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:11.683103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 159/329] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:11.693895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999248f-0a12-4a81-8ba3-c26844e4833a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:11.734195 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4630a6-0352-444d-883d-7012c1ecf4c6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:11.783333 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf45549-9c8b-4175-988b-29ef77037fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:11.784647 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf45549-9c8b-4175-988b-29ef77037fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:11.785205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf45549-9c8b-4175-988b-29ef77037fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:11.785679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf45549-9c8b-4175-988b-29ef77037fa8 None admin] PATH_INFO: `/v3/users/a9f619bf1e444fbe8455ce5171a8bb30` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:11.787306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf45549-9c8b-4175-988b-29ef77037fa8 None admin] RBAC: Authorizing `identity:get_user(user_id=a9f619bf1e444fbe8455ce5171a8bb30)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:11.790021 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf45549-9c8b-4175-988b-29ef77037fa8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:11.792690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 171/330] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:11 2022] GET /identity/v3/users/a9f619bf1e444fbe8455ce5171a8bb30 => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:11.797908 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf45549-9c8b-4175-988b-29ef77037fa8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:11.834845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999248f-0a12-4a81-8ba3-c26844e4833a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:11.863875 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:11.878748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b336c-b70d-4a49-9e2b-cd5a8c14276c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:11.879853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b336c-b70d-4a49-9e2b-cd5a8c14276c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:11.880274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b336c-b70d-4a49-9e2b-cd5a8c14276c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:11.880622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188b336c-b70d-4a49-9e2b-cd5a8c14276c None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:11.881393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b336c-b70d-4a49-9e2b-cd5a8c14276c None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:11.881752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b336c-b70d-4a49-9e2b-cd5a8c14276c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:11.883241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188b336c-b70d-4a49-9e2b-cd5a8c14276c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:11.892955 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 160/331] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:11.899334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b336c-b70d-4a49-9e2b-cd5a8c14276c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:11.920336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf45549-9c8b-4175-988b-29ef77037fa8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:11.937954 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:11.957681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19411889-0f5e-4e10-baf9-4ed32c8afc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:11.958640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19411889-0f5e-4e10-baf9-4ed32c8afc99 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:11.959098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19411889-0f5e-4e10-baf9-4ed32c8afc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:11.959546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19411889-0f5e-4e10-baf9-4ed32c8afc99 None admin] PATH_INFO: `/v3/system/users/a9f619bf1e444fbe8455ce5171a8bb30/roles/6a413519e66945239f17395ca12293c2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:11.961376 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19411889-0f5e-4e10-baf9-4ed32c8afc99 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a9f619bf1e444fbe8455ce5171a8bb30, role_id=6a413519e66945239f17395ca12293c2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:11.963731 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19411889-0f5e-4e10-baf9-4ed32c8afc99 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:12.022975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188b336c-b70d-4a49-9e2b-cd5a8c14276c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:12.029811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 172/332] 10.222.0.67 () {58 vars in 1334 bytes} [Mon Sep 26 17:44:11 2022] PUT /identity/v3/system/users/a9f619bf1e444fbe8455ce5171a8bb30/roles/6a413519e66945239f17395ca12293c2 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 17:44:12.062689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db77d72-4e9c-4273-9643-2ba8e6fd199d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:12.063270 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db77d72-4e9c-4273-9643-2ba8e6fd199d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:12.063554 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db77d72-4e9c-4273-9643-2ba8e6fd199d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:12.063939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db77d72-4e9c-4273-9643-2ba8e6fd199d None admin] PATH_INFO: `/v3/users/c54bb29c001640a3beebd69641326465` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:12.065317 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db77d72-4e9c-4273-9643-2ba8e6fd199d None admin] RBAC: Authorizing `identity:get_user(user_id=c54bb29c001640a3beebd69641326465)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:12.066989 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db77d72-4e9c-4273-9643-2ba8e6fd199d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:12.071196 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 161/333] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:11 2022] GET /identity/v3/users/c54bb29c001640a3beebd69641326465 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:12.075284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19411889-0f5e-4e10-baf9-4ed32c8afc99 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:12.197471 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19411889-0f5e-4e10-baf9-4ed32c8afc99 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:12.221331 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:12.243150 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff7c92-436c-43ad-80e8-99a41db0847f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:12.243823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff7c92-436c-43ad-80e8-99a41db0847f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:12.244109 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff7c92-436c-43ad-80e8-99a41db0847f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:12.244388 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff7c92-436c-43ad-80e8-99a41db0847f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:12.245212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff7c92-436c-43ad-80e8-99a41db0847f None admin] RBAC: Adding query filter params (role.id=1c983c7e32e44c61be3996c28399e617, user.id=c54bb29c001640a3beebd69641326465, scope.system=all) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:12.245526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff7c92-436c-43ad-80e8-99a41db0847f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:12.247028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff7c92-436c-43ad-80e8-99a41db0847f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:12.264594 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 173/334] 10.222.0.67 () {56 vars in 1308 bytes} [Mon Sep 26 17:44:12 2022] GET /identity/v3/role_assignments?user.id=c54bb29c001640a3beebd69641326465&scope.system=all&role.id=1c983c7e32e44c61be3996c28399e617 => generated 510 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:14.313331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba254b-f4cd-4333-bc2b-793cf04d8f61 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:14.313841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba254b-f4cd-4333-bc2b-793cf04d8f61 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:14.313841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ba254b-f4cd-4333-bc2b-793cf04d8f61 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:14.315399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 162/335] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:14.322434 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe123fb-4512-4eb1-a8ca-bcbc04e83200 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:14.322876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe123fb-4512-4eb1-a8ca-bcbc04e83200 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:14.323074 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe123fb-4512-4eb1-a8ca-bcbc04e83200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:14.342122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c8b2f-5af9-4c47-88b3-847dc8234d50 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:14.342415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c8b2f-5af9-4c47-88b3-847dc8234d50 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:14.342678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3c8b2f-5af9-4c47-88b3-847dc8234d50 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:14.344296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 163/336] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:44:14.349439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6fe123fb-4512-4eb1-a8ca-bcbc04e83200 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:14.350341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b1cee-a346-4628-8538-60f31f5311e9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:14.350649 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6fe123fb-4512-4eb1-a8ca-bcbc04e83200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:14.351025 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b1cee-a346-4628-8538-60f31f5311e9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:14.351505 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b1cee-a346-4628-8538-60f31f5311e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:14.379012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-257b1cee-a346-4628-8538-60f31f5311e9 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:14.380644 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-257b1cee-a346-4628-8538-60f31f5311e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:14.474433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 174/337] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:14.514619 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 164/338] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:14.712653 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00f7fd-91bb-47ba-b64c-7702f2cb3b0c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:14.712981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00f7fd-91bb-47ba-b64c-7702f2cb3b0c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:14.713220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c00f7fd-91bb-47ba-b64c-7702f2cb3b0c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:14.714839 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 175/339] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:14.720448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d292a1-82a7-4b5c-839c-a6a328f3f2e1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:14.720814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d292a1-82a7-4b5c-839c-a6a328f3f2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:14.721082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d292a1-82a7-4b5c-839c-a6a328f3f2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:14.725936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f68886-076b-4f96-af0a-094dcec9e387 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:14.726218 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f68886-076b-4f96-af0a-094dcec9e387 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:14.726496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f68886-076b-4f96-af0a-094dcec9e387 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:14.730041 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 176/340] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:14.736720 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4072440-a675-4607-9104-c91be460f37c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:14.737003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4072440-a675-4607-9104-c91be460f37c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:14.737286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4072440-a675-4607-9104-c91be460f37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:14.748048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d4d292a1-82a7-4b5c-839c-a6a328f3f2e1 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:14.749567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d4d292a1-82a7-4b5c-839c-a6a328f3f2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:14.773218 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a4072440-a675-4607-9104-c91be460f37c None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:14.774690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a4072440-a675-4607-9104-c91be460f37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:14.824656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:14.897849 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 177/341] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:14.921481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 165/342] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:14.925458 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4072440-a675-4607-9104-c91be460f37c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:14.959744 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:15.031096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4072440-a675-4607-9104-c91be460f37c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:15.071154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9797706-caf3-4d2f-b1c0-e7367b7b9145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:15.071820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9797706-caf3-4d2f-b1c0-e7367b7b9145 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:15.072097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9797706-caf3-4d2f-b1c0-e7367b7b9145 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:15.072363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9797706-caf3-4d2f-b1c0-e7367b7b9145 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:15.085158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9797706-caf3-4d2f-b1c0-e7367b7b9145 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f9797706-caf3-4d2f-b1c0-e7367b7b9145 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:15.089991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:15.090836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:15.091619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:15.091619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:15.091619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:15.091619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:15.091619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:15.091619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:15.091619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:15.091619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:15.091619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:15.091619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:15.091619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:15.091619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:15.091619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 178/343] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:44:15.097699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d292a1-82a7-4b5c-839c-a6a328f3f2e1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:15.224715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9797706-caf3-4d2f-b1c0-e7367b7b9145 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:15.230242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d292a1-82a7-4b5c-839c-a6a328f3f2e1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:15.283750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5910a6-7c04-40d0-9ca9-c8aefa5475a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:15.284914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5910a6-7c04-40d0-9ca9-c8aefa5475a1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:15.285215 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5910a6-7c04-40d0-9ca9-c8aefa5475a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:15.285681 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5910a6-7c04-40d0-9ca9-c8aefa5475a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:15.286378 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5910a6-7c04-40d0-9ca9-c8aefa5475a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:15.286687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5910a6-7c04-40d0-9ca9-c8aefa5475a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:15.288126 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5910a6-7c04-40d0-9ca9-c8aefa5475a1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:15.297670 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 166/344] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:15.316972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5910a6-7c04-40d0-9ca9-c8aefa5475a1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:15.367122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:15.370887 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:15.379868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9797706-caf3-4d2f-b1c0-e7367b7b9145 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:15.424423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664c068-e198-4213-bad6-9e5f9f7cafbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:15.425488 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664c068-e198-4213-bad6-9e5f9f7cafbf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:15.425936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664c068-e198-4213-bad6-9e5f9f7cafbf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:15.426444 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1664c068-e198-4213-bad6-9e5f9f7cafbf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:15.445687 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1664c068-e198-4213-bad6-9e5f9f7cafbf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1664c068-e198-4213-bad6-9e5f9f7cafbf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:15.450970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:15.452034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:15.452850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:15.452850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:15.452850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:15.452850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:15.452850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:15.452850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:15.452850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:15.452850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:15.452850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:15.452850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:15.452850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:15.452850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:15.454925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 179/345] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:44:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:15.462514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664c068-e198-4213-bad6-9e5f9f7cafbf None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:15.495367 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5910a6-7c04-40d0-9ca9-c8aefa5475a1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:15.572565 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bf858-a46e-47a5-a16e-d87ab62f9d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:15.573189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bf858-a46e-47a5-a16e-d87ab62f9d84 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:15.573307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bf858-a46e-47a5-a16e-d87ab62f9d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:15.573546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451bf858-a46e-47a5-a16e-d87ab62f9d84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:15.586416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1664c068-e198-4213-bad6-9e5f9f7cafbf None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:15.627991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f62a4d-f47b-4f31-a043-f28467eec246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:15.629214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f62a4d-f47b-4f31-a043-f28467eec246 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:15.629676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f62a4d-f47b-4f31-a043-f28467eec246 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:15.630153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f62a4d-f47b-4f31-a043-f28467eec246 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:15.631041 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f62a4d-f47b-4f31-a043-f28467eec246 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:15.632020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f62a4d-f47b-4f31-a043-f28467eec246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:15.633809 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f62a4d-f47b-4f31-a043-f28467eec246 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:15.647772 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 180/346] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:15.648977 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f62a4d-f47b-4f31-a043-f28467eec246 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:15.661125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bf858-a46e-47a5-a16e-d87ab62f9d84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:15.663213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451bf858-a46e-47a5-a16e-d87ab62f9d84 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:15.671934 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:15.697008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:15.764755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f62a4d-f47b-4f31-a043-f28467eec246 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:15.838771 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d50eb-d270-46b1-91e2-53ab1b0d3551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:15.839337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d50eb-d270-46b1-91e2-53ab1b0d3551 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:15.839619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d50eb-d270-46b1-91e2-53ab1b0d3551 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:15.839980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d50eb-d270-46b1-91e2-53ab1b0d3551 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:15.848547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 167/347] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:15.851339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bf858-a46e-47a5-a16e-d87ab62f9d84 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:15.862588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d50eb-d270-46b1-91e2-53ab1b0d3551 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-643d50eb-d270-46b1-91e2-53ab1b0d3551 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:15.869662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:15.870636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:15.871435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:15.871435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:15.871435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:15.871435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:15.871435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:15.871435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:15.871435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:15.871435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:15.871435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:15.871435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:15.871435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 26 17:44:15.871435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:15.873728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 181/348] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:15.874702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d50eb-d270-46b1-91e2-53ab1b0d3551 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:16.061069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451bf858-a46e-47a5-a16e-d87ab62f9d84 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:16.074428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:16.097482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:16.110127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d50eb-d270-46b1-91e2-53ab1b0d3551 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:16.111066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c50e9-90f1-4129-93cd-06540751c12c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:16.112007 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c50e9-90f1-4129-93cd-06540751c12c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:16.112577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c50e9-90f1-4129-93cd-06540751c12c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:16.113721 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90c50e9-90f1-4129-93cd-06540751c12c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:16.134653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90c50e9-90f1-4129-93cd-06540751c12c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f90c50e9-90f1-4129-93cd-06540751c12c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:16.141027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:16.145993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:16.146762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:16.146762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:16.146762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:16.146762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:16.146762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:16.146762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:16.146762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:16.146762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:16.146762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:16.146762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:16.146762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:16.146762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:16.146762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 168/349] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:16.146762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c50e9-90f1-4129-93cd-06540751c12c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:16.167341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cde36-174c-4510-8ce6-0d0d4c11d062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:16.168588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cde36-174c-4510-8ce6-0d0d4c11d062 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:16.170587 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cde36-174c-4510-8ce6-0d0d4c11d062 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:16.171262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786cde36-174c-4510-8ce6-0d0d4c11d062 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:16.279690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786cde36-174c-4510-8ce6-0d0d4c11d062 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:16.281644 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786cde36-174c-4510-8ce6-0d0d4c11d062 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:16.298413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90c50e9-90f1-4129-93cd-06540751c12c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:16.350415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77992ff-5c03-4bce-9a55-ac8f02c8a83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:16.351011 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77992ff-5c03-4bce-9a55-ac8f02c8a83d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:16.351296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77992ff-5c03-4bce-9a55-ac8f02c8a83d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:16.351615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77992ff-5c03-4bce-9a55-ac8f02c8a83d None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:16.352829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77992ff-5c03-4bce-9a55-ac8f02c8a83d None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:16.354278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77992ff-5c03-4bce-9a55-ac8f02c8a83d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:16.355956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 169/350] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:16 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:16.357688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77992ff-5c03-4bce-9a55-ac8f02c8a83d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:16.400128 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:16.430575 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:16.436444 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 182/351] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:16.442079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cde36-174c-4510-8ce6-0d0d4c11d062 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:16.478287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77992ff-5c03-4bce-9a55-ac8f02c8a83d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:16.521036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5080cbb0-d990-42ec-95ff-40aa599fd554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:16.521650 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080cbb0-d990-42ec-95ff-40aa599fd554 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:16.521925 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080cbb0-d990-42ec-95ff-40aa599fd554 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:16.522194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5080cbb0-d990-42ec-95ff-40aa599fd554 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:16.522876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080cbb0-d990-42ec-95ff-40aa599fd554 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:16.523159 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080cbb0-d990-42ec-95ff-40aa599fd554 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:16.524583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5080cbb0-d990-42ec-95ff-40aa599fd554 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:16.536165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 170/352] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:16.536588 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719393ac-8d2e-4d87-bd27-bbb162cdb065 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:16.536798 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719393ac-8d2e-4d87-bd27-bbb162cdb065 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:16.537072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719393ac-8d2e-4d87-bd27-bbb162cdb065 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:16.538438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 171/353] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:16.539142 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719393ac-8d2e-4d87-bd27-bbb162cdb065 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:16.577276 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786cde36-174c-4510-8ce6-0d0d4c11d062 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:16.618767 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54965d-c86e-4834-9853-8991e2e6ebc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:16.619386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54965d-c86e-4834-9853-8991e2e6ebc4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:16.619687 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54965d-c86e-4834-9853-8991e2e6ebc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:16.619975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54965d-c86e-4834-9853-8991e2e6ebc4 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:16.620630 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54965d-c86e-4834-9853-8991e2e6ebc4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:16.620932 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54965d-c86e-4834-9853-8991e2e6ebc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:16.622289 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f54965d-c86e-4834-9853-8991e2e6ebc4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:16.631678 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 183/354] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:16.632503 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54965d-c86e-4834-9853-8991e2e6ebc4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:16.673669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719393ac-8d2e-4d87-bd27-bbb162cdb065 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:16.683462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:16.696214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:16.716547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b13c4b2-76b6-447e-822b-b9e634411d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:16.717185 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13c4b2-76b6-447e-822b-b9e634411d6f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:16.717464 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13c4b2-76b6-447e-822b-b9e634411d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:16.717733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b13c4b2-76b6-447e-822b-b9e634411d6f None admin] PATH_INFO: `/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:16.726167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13c4b2-76b6-447e-822b-b9e634411d6f None admin] RBAC: Authorizing `identity:get_group(group_id=6e3090dc484c4fc39d1d9ca6fbdeae29)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:16.727612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b13c4b2-76b6-447e-822b-b9e634411d6f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:16.729463 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 172/355] 10.222.0.67 () {56 vars in 1180 bytes} [Mon Sep 26 17:44:16 2022] GET /identity/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:16.730753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fab6a2-615e-466f-b110-1c54d523ecf4 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:16.731869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fab6a2-615e-466f-b110-1c54d523ecf4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:16.731869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fab6a2-615e-466f-b110-1c54d523ecf4 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:16.733855 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 173/356] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:16.734997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66286e92-67ed-473d-bcbe-260013829f05 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:16.735279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66286e92-67ed-473d-bcbe-260013829f05 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:16.735682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66286e92-67ed-473d-bcbe-260013829f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:16.755084 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-66286e92-67ed-473d-bcbe-260013829f05 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:16.756377 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-66286e92-67ed-473d-bcbe-260013829f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:16.756569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54965d-c86e-4834-9853-8991e2e6ebc4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:16.803481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87259e-9bda-468d-a1cf-1ee40e212b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:16.804232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87259e-9bda-468d-a1cf-1ee40e212b96 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:16.804510 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87259e-9bda-468d-a1cf-1ee40e212b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:16.804783 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d87259e-9bda-468d-a1cf-1ee40e212b96 None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:16.805910 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87259e-9bda-468d-a1cf-1ee40e212b96 None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:16.807326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d87259e-9bda-468d-a1cf-1ee40e212b96 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:16.808991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 184/357] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:16 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:16.809734 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87259e-9bda-468d-a1cf-1ee40e212b96 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:16.840832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 174/358] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:16.842040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66286e92-67ed-473d-bcbe-260013829f05 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:16.935014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d87259e-9bda-468d-a1cf-1ee40e212b96 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:16.946396 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:16.968254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66286e92-67ed-473d-bcbe-260013829f05 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:16.979673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68aa30-51b3-497b-8409-343a3de44057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:16.979673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68aa30-51b3-497b-8409-343a3de44057 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:16.979673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68aa30-51b3-497b-8409-343a3de44057 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:16.979673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68aa30-51b3-497b-8409-343a3de44057 None admin] PATH_INFO: `/v3/users/a9f619bf1e444fbe8455ce5171a8bb30` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:16.980054 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68aa30-51b3-497b-8409-343a3de44057 None admin] RBAC: Authorizing `identity:get_user(user_id=a9f619bf1e444fbe8455ce5171a8bb30)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:16.980509 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b68aa30-51b3-497b-8409-343a3de44057 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:16.982403 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 185/359] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:44:16 2022] GET /identity/v3/users/a9f619bf1e444fbe8455ce5171a8bb30 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:16.984082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae009c89-6dff-4bea-b762-a015b2df5996 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:16.984431 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae009c89-6dff-4bea-b762-a015b2df5996 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:16.984776 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae009c89-6dff-4bea-b762-a015b2df5996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:16.994660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:17.008377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ae009c89-6dff-4bea-b762-a015b2df5996 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:17.009809 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ae009c89-6dff-4bea-b762-a015b2df5996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:17.022052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533c9fa-205b-4be2-a02f-2b55f4444f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:17.022786 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533c9fa-205b-4be2-a02f-2b55f4444f06 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:17.023069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533c9fa-205b-4be2-a02f-2b55f4444f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:17.023343 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533c9fa-205b-4be2-a02f-2b55f4444f06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:17.111671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 186/360] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:17.111671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae009c89-6dff-4bea-b762-a015b2df5996 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:17.128000 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533c9fa-205b-4be2-a02f-2b55f4444f06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:17.129811 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533c9fa-205b-4be2-a02f-2b55f4444f06 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:17.240376 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae009c89-6dff-4bea-b762-a015b2df5996 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:17.249390 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:17.255308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 175/361] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:17.257497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533c9fa-205b-4be2-a02f-2b55f4444f06 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:17.276424 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:17.289730 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361862d-3cda-4f3c-ac4a-2e5656569c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:17.290663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361862d-3cda-4f3c-ac4a-2e5656569c6f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:17.290947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361862d-3cda-4f3c-ac4a-2e5656569c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:17.291224 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361862d-3cda-4f3c-ac4a-2e5656569c6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:17.291985 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361862d-3cda-4f3c-ac4a-2e5656569c6f None admin] RBAC: Adding query filter params (group.id=6e3090dc484c4fc39d1d9ca6fbdeae29, role.id=1c983c7e32e44c61be3996c28399e617, scope.project.id=25808aef0f69482dbf49592f737cff01) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:17.292295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361862d-3cda-4f3c-ac4a-2e5656569c6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:17.294002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1361862d-3cda-4f3c-ac4a-2e5656569c6f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:17.299765 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 187/362] 10.222.0.67 () {56 vars in 1376 bytes} [Mon Sep 26 17:44:17 2022] GET /identity/v3/role_assignments?group.id=6e3090dc484c4fc39d1d9ca6fbdeae29&scope.project.id=25808aef0f69482dbf49592f737cff01&role.id=1c983c7e32e44c61be3996c28399e617 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:17.302110 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361862d-3cda-4f3c-ac4a-2e5656569c6f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:17.385944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533c9fa-205b-4be2-a02f-2b55f4444f06 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:17.428938 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361862d-3cda-4f3c-ac4a-2e5656569c6f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:17.430534 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2d11e-4c3d-462b-8dd9-69024c6a8ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:17.431267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2d11e-4c3d-462b-8dd9-69024c6a8ad2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:17.431658 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2d11e-4c3d-462b-8dd9-69024c6a8ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:17.431955 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2d11e-4c3d-462b-8dd9-69024c6a8ad2 None admin] PATH_INFO: `/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:17.436452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2d11e-4c3d-462b-8dd9-69024c6a8ad2 None admin] RBAC: Authorizing `identity:get_group(group_id=6e3090dc484c4fc39d1d9ca6fbdeae29)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:17.438195 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2d11e-4c3d-462b-8dd9-69024c6a8ad2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:17.439991 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 176/363] 10.222.0.67 () {56 vars in 1180 bytes} [Mon Sep 26 17:44:17 2022] GET /identity/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:17.441575 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2d11e-4c3d-462b-8dd9-69024c6a8ad2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:17.478643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bf249-d492-4ed0-a102-5c7f25b5f240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:17.479704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bf249-d492-4ed0-a102-5c7f25b5f240 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:17.479704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bf249-d492-4ed0-a102-5c7f25b5f240 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:17.479947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39bf249-d492-4ed0-a102-5c7f25b5f240 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:17.480730 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bf249-d492-4ed0-a102-5c7f25b5f240 None admin] RBAC: Adding query filter params (role.id=6a413519e66945239f17395ca12293c2, user.id=a9f619bf1e444fbe8455ce5171a8bb30, scope.system=all) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:17.481019 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bf249-d492-4ed0-a102-5c7f25b5f240 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:17.482574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39bf249-d492-4ed0-a102-5c7f25b5f240 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:17.496330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 188/364] 10.222.0.67 () {56 vars in 1308 bytes} [Mon Sep 26 17:44:17 2022] GET /identity/v3/role_assignments?user.id=a9f619bf1e444fbe8455ce5171a8bb30&scope.system=all&role.id=6a413519e66945239f17395ca12293c2 => generated 510 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:17.504891 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bf249-d492-4ed0-a102-5c7f25b5f240 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:17.543735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:17.544504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:17.580877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2d11e-4c3d-462b-8dd9-69024c6a8ad2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:17.635041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf8019-9b54-44e9-a0a7-31668871b6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:17.636308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf8019-9b54-44e9-a0a7-31668871b6ad None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:17.636825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf8019-9b54-44e9-a0a7-31668871b6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:17.637627 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadf8019-9b54-44e9-a0a7-31668871b6ad None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:17.653238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadf8019-9b54-44e9-a0a7-31668871b6ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:17.655893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39bf249-d492-4ed0-a102-5c7f25b5f240 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-eadf8019-9b54-44e9-a0a7-31668871b6ad None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:17.661180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:17.662115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:17.662940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:17.662940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:17.662940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:17.662940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:17.662940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:17.662940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:17.662940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:17.662940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:17.662940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:17.662940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:17.662940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:17.662940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:17.665985 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 177/365] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:44:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:17.667898 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf8019-9b54-44e9-a0a7-31668871b6ad None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:17.716118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a207cf-aaae-4179-8713-0df45bb10d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:17.716766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a207cf-aaae-4179-8713-0df45bb10d8e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:17.717045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a207cf-aaae-4179-8713-0df45bb10d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:17.717316 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a207cf-aaae-4179-8713-0df45bb10d8e None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:17.720998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a207cf-aaae-4179-8713-0df45bb10d8e None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:17.722275 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a207cf-aaae-4179-8713-0df45bb10d8e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:17.724294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 189/366] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:17 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:17.726731 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a207cf-aaae-4179-8713-0df45bb10d8e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:17.815342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadf8019-9b54-44e9-a0a7-31668871b6ad None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:17.851059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:17.857165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f538aa3-f1fd-43e2-8d87-a98a55988f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:17.857801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f538aa3-f1fd-43e2-8d87-a98a55988f76 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:17.858080 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f538aa3-f1fd-43e2-8d87-a98a55988f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:17.858252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a207cf-aaae-4179-8713-0df45bb10d8e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:17.858346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f538aa3-f1fd-43e2-8d87-a98a55988f76 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:17.867469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:17.872778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f538aa3-f1fd-43e2-8d87-a98a55988f76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5f538aa3-f1fd-43e2-8d87-a98a55988f76 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:17.877757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:17.878849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:17.880720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:17.880720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:17.880720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:17.880720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:17.880720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:17.880720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:17.880720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:17.880720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:17.880720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:17.880720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:17.880720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:17.880720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:17.880720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 178/367] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:44:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:17.880720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f538aa3-f1fd-43e2-8d87-a98a55988f76 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:17.936601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6800945-8e2b-4b31-a7dc-dcda2ac8779e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:17.937568 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6800945-8e2b-4b31-a7dc-dcda2ac8779e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:17.938027 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6800945-8e2b-4b31-a7dc-dcda2ac8779e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:17.938476 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6800945-8e2b-4b31-a7dc-dcda2ac8779e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:17.939403 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6800945-8e2b-4b31-a7dc-dcda2ac8779e None admin] RBAC: Adding query filter params (group.id=6e3090dc484c4fc39d1d9ca6fbdeae29, role.id=30bb990dfe5c45fbbfed21c08c7859b4, scope.project.id=25808aef0f69482dbf49592f737cff01) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:17.939738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6800945-8e2b-4b31-a7dc-dcda2ac8779e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:17.942164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6800945-8e2b-4b31-a7dc-dcda2ac8779e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:17.948617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 190/368] 10.222.0.67 () {56 vars in 1376 bytes} [Mon Sep 26 17:44:17 2022] GET /identity/v3/role_assignments?group.id=6e3090dc484c4fc39d1d9ca6fbdeae29&scope.project.id=25808aef0f69482dbf49592f737cff01&role.id=30bb990dfe5c45fbbfed21c08c7859b4 => generated 260 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:17.949260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6800945-8e2b-4b31-a7dc-dcda2ac8779e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:18.029844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f538aa3-f1fd-43e2-8d87-a98a55988f76 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:18.080420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329db30-d31d-487a-981b-8a44120a2469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:18.081491 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329db30-d31d-487a-981b-8a44120a2469 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:18.082281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329db30-d31d-487a-981b-8a44120a2469 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:18.083173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c329db30-d31d-487a-981b-8a44120a2469 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:18.084083 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c329db30-d31d-487a-981b-8a44120a2469 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:18.084599 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c329db30-d31d-487a-981b-8a44120a2469 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:18.086196 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c329db30-d31d-487a-981b-8a44120a2469 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:18.100235 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 179/369] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:18.101440 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329db30-d31d-487a-981b-8a44120a2469 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:18.108638 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6800945-8e2b-4b31-a7dc-dcda2ac8779e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:18.154420 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e590a-4d6d-4a80-b9f9-33254ab25213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:18.155224 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e590a-4d6d-4a80-b9f9-33254ab25213 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:18.155580 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e590a-4d6d-4a80-b9f9-33254ab25213 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:18.155975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e590a-4d6d-4a80-b9f9-33254ab25213 None admin] PATH_INFO: `/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:18.157954 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e590a-4d6d-4a80-b9f9-33254ab25213 None admin] RBAC: Authorizing `identity:get_group(group_id=6e3090dc484c4fc39d1d9ca6fbdeae29)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:18.160282 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e590a-4d6d-4a80-b9f9-33254ab25213 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:18.163006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 191/370] 10.222.0.67 () {56 vars in 1180 bytes} [Mon Sep 26 17:44:17 2022] GET /identity/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29 => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:18.163393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e590a-4d6d-4a80-b9f9-33254ab25213 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:18.190144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:18.203062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:18.258597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c329db30-d31d-487a-981b-8a44120a2469 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:18.310731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cf9ea-97bf-4d9d-b123-75cb6b1aa616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:18.311351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cf9ea-97bf-4d9d-b123-75cb6b1aa616 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:18.311641 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cf9ea-97bf-4d9d-b123-75cb6b1aa616 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:18.311894 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e590a-4d6d-4a80-b9f9-33254ab25213 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:18.312083 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14cf9ea-97bf-4d9d-b123-75cb6b1aa616 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:18.312595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cf9ea-97bf-4d9d-b123-75cb6b1aa616 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:18.312898 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cf9ea-97bf-4d9d-b123-75cb6b1aa616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:18.314293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14cf9ea-97bf-4d9d-b123-75cb6b1aa616 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:18.326560 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 180/371] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:18.327431 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cf9ea-97bf-4d9d-b123-75cb6b1aa616 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:18.363584 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779308cb-f757-40f7-91f6-63ab23385d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:18.364681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779308cb-f757-40f7-91f6-63ab23385d28 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:18.364960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779308cb-f757-40f7-91f6-63ab23385d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:18.365515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779308cb-f757-40f7-91f6-63ab23385d28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:18.454546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14cf9ea-97bf-4d9d-b123-75cb6b1aa616 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:18.457631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779308cb-f757-40f7-91f6-63ab23385d28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:18.459914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779308cb-f757-40f7-91f6-63ab23385d28 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:18.488081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:18.496796 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:18.497340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed74c0-22bf-4d8b-ad36-48ed577e38c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:18.498077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed74c0-22bf-4d8b-ad36-48ed577e38c6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:18.498329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed74c0-22bf-4d8b-ad36-48ed577e38c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:18.498607 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ed74c0-22bf-4d8b-ad36-48ed577e38c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:18.500269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed74c0-22bf-4d8b-ad36-48ed577e38c6 None admin] RBAC: Adding query filter params (group.id=6e3090dc484c4fc39d1d9ca6fbdeae29, role.id=1c983c7e32e44c61be3996c28399e617, scope.project.id=887965a382184db9958ad175261f03a1) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:18.500582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed74c0-22bf-4d8b-ad36-48ed577e38c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:18.502155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ed74c0-22bf-4d8b-ad36-48ed577e38c6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:18.509996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 181/372] 10.222.0.67 () {56 vars in 1376 bytes} [Mon Sep 26 17:44:18 2022] GET /identity/v3/role_assignments?group.id=6e3090dc484c4fc39d1d9ca6fbdeae29&scope.project.id=887965a382184db9958ad175261f03a1&role.id=1c983c7e32e44c61be3996c28399e617 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:18.510871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed74c0-22bf-4d8b-ad36-48ed577e38c6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:18.606048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 192/373] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:18.608763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779308cb-f757-40f7-91f6-63ab23385d28 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:18.644474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ed74c0-22bf-4d8b-ad36-48ed577e38c6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:18.686344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04363b34-4342-40b2-a2d9-ee3de8d251a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:18.687757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04363b34-4342-40b2-a2d9-ee3de8d251a0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:18.688118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04363b34-4342-40b2-a2d9-ee3de8d251a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:18.688679 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04363b34-4342-40b2-a2d9-ee3de8d251a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:18.770067 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779308cb-f757-40f7-91f6-63ab23385d28 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:18.802400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:18.817878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:18.820451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04363b34-4342-40b2-a2d9-ee3de8d251a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:18.823533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04363b34-4342-40b2-a2d9-ee3de8d251a0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:18.834001 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e24a93-ed15-4200-970b-79fe967d1638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:18.834681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e24a93-ed15-4200-970b-79fe967d1638 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:18.838264 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e24a93-ed15-4200-970b-79fe967d1638 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:18.839027 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e24a93-ed15-4200-970b-79fe967d1638 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:18.840447 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e24a93-ed15-4200-970b-79fe967d1638 None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:18.842399 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e24a93-ed15-4200-970b-79fe967d1638 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:18.844440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 193/374] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:18 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:18.851219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e24a93-ed15-4200-970b-79fe967d1638 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:18.966888 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e24a93-ed15-4200-970b-79fe967d1638 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:18.991102 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 182/375] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:19.000871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04363b34-4342-40b2-a2d9-ee3de8d251a0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:19.019531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1f6aa-ca7f-4c32-a0a6-b2b044d8b791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:19.020368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1f6aa-ca7f-4c32-a0a6-b2b044d8b791 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:19.020651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1f6aa-ca7f-4c32-a0a6-b2b044d8b791 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:19.020927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1f6aa-ca7f-4c32-a0a6-b2b044d8b791 None admin] PATH_INFO: `/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:19.028616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1f6aa-ca7f-4c32-a0a6-b2b044d8b791 None admin] RBAC: Authorizing `identity:get_group(group_id=6e3090dc484c4fc39d1d9ca6fbdeae29)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:19.030236 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1f6aa-ca7f-4c32-a0a6-b2b044d8b791 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:19.032247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 194/376] 10.222.0.67 () {56 vars in 1180 bytes} [Mon Sep 26 17:44:18 2022] GET /identity/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:19.040300 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1f6aa-ca7f-4c32-a0a6-b2b044d8b791 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:19.110037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:19.130033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04363b34-4342-40b2-a2d9-ee3de8d251a0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:19.155640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:19.200623 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3e67a-86e1-4dcf-86c1-2c1c0de8f883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:19.201342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3e67a-86e1-4dcf-86c1-2c1c0de8f883 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:19.201627 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3e67a-86e1-4dcf-86c1-2c1c0de8f883 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:19.201901 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d3e67a-86e1-4dcf-86c1-2c1c0de8f883 None admin] PATH_INFO: `/v3/projects/1537d57f83734659b1130d2661d44ac2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:19.203030 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3e67a-86e1-4dcf-86c1-2c1c0de8f883 None admin] RBAC: Authorizing `identity:get_project(project_id=1537d57f83734659b1130d2661d44ac2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:19.204499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d3e67a-86e1-4dcf-86c1-2c1c0de8f883 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:19.206988 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 183/377] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:18 2022] GET /identity/v3/projects/1537d57f83734659b1130d2661d44ac2 => generated 344 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:19.212289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3e67a-86e1-4dcf-86c1-2c1c0de8f883 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:19.217925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1f6aa-ca7f-4c32-a0a6-b2b044d8b791 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:19.280606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5802db9-fd49-4dbb-a91e-68e8b3af6281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:19.281223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5802db9-fd49-4dbb-a91e-68e8b3af6281 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:19.281504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5802db9-fd49-4dbb-a91e-68e8b3af6281 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:19.281780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5802db9-fd49-4dbb-a91e-68e8b3af6281 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:19.282505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5802db9-fd49-4dbb-a91e-68e8b3af6281 None admin] RBAC: Adding query filter params (group.id=6e3090dc484c4fc39d1d9ca6fbdeae29, role.id=30bb990dfe5c45fbbfed21c08c7859b4, scope.project.id=887965a382184db9958ad175261f03a1) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:19.282814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5802db9-fd49-4dbb-a91e-68e8b3af6281 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:19.284820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5802db9-fd49-4dbb-a91e-68e8b3af6281 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:19.299423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 195/378] 10.222.0.67 () {56 vars in 1376 bytes} [Mon Sep 26 17:44:19 2022] GET /identity/v3/role_assignments?group.id=6e3090dc484c4fc39d1d9ca6fbdeae29&scope.project.id=887965a382184db9958ad175261f03a1&role.id=30bb990dfe5c45fbbfed21c08c7859b4 => generated 260 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:19.396912 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d3e67a-86e1-4dcf-86c1-2c1c0de8f883 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:19.446524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fbb24-7f45-40e9-9875-309bb3c7fd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:19.447194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fbb24-7f45-40e9-9875-309bb3c7fd08 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:19.447471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fbb24-7f45-40e9-9875-309bb3c7fd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:19.447763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fbb24-7f45-40e9-9875-309bb3c7fd08 None admin] PATH_INFO: `/v3/groups/a3b437bb95304f75a3a95b6b679a65cb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:19.453414 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fbb24-7f45-40e9-9875-309bb3c7fd08 None admin] RBAC: Authorizing `identity:get_group(group_id=a3b437bb95304f75a3a95b6b679a65cb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:19.454933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fbb24-7f45-40e9-9875-309bb3c7fd08 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:19.457966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 184/379] 10.222.0.67 () {56 vars in 1180 bytes} [Mon Sep 26 17:44:19 2022] GET /identity/v3/groups/a3b437bb95304f75a3a95b6b679a65cb => generated 229 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:19.463631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5802db9-fd49-4dbb-a91e-68e8b3af6281 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:19.593594 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5802db9-fd49-4dbb-a91e-68e8b3af6281 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:19.626144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:19.638652 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e455ed-1c81-4f47-a595-ef339f1b43d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:19.644642 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e455ed-1c81-4f47-a595-ef339f1b43d4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:19.644642 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e455ed-1c81-4f47-a595-ef339f1b43d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:19.644642 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e455ed-1c81-4f47-a595-ef339f1b43d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:19.644642 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e455ed-1c81-4f47-a595-ef339f1b43d4 None admin] RBAC: Adding query filter params (group.id=a3b437bb95304f75a3a95b6b679a65cb, role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=1537d57f83734659b1130d2661d44ac2) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:19.644642 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e455ed-1c81-4f47-a595-ef339f1b43d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:19.644642 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e455ed-1c81-4f47-a595-ef339f1b43d4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:19.651580 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 196/380] 10.222.0.67 () {56 vars in 1376 bytes} [Mon Sep 26 17:44:19 2022] GET /identity/v3/role_assignments?group.id=a3b437bb95304f75a3a95b6b679a65cb&scope.project.id=1537d57f83734659b1130d2661d44ac2&role.id=07626899472b4e32adc0c71ce796a25d => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:20.017741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb656f-1c9c-4c39-b545-e9dc8b862174 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:20.018077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb656f-1c9c-4c39-b545-e9dc8b862174 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:20.018275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cb656f-1c9c-4c39-b545-e9dc8b862174 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:20.019894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 185/381] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:20.025335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe84ec-a8b3-4ae3-aea4-2a363d2e03c5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:20.025626 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe84ec-a8b3-4ae3-aea4-2a363d2e03c5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:20.025900 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbe84ec-a8b3-4ae3-aea4-2a363d2e03c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:20.047562 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bbbe84ec-a8b3-4ae3-aea4-2a363d2e03c5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:20.048774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bbbe84ec-a8b3-4ae3-aea4-2a363d2e03c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:20.145582 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 197/382] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:20.483942 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c23f21-0531-4bfc-9c08-a4dcffad90a8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:20.484480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c23f21-0531-4bfc-9c08-a4dcffad90a8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:20.484480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c23f21-0531-4bfc-9c08-a4dcffad90a8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:20.486010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 186/383] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:20.491933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4fa92-bf7a-447b-9b20-813c2067180f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:20.492752 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4fa92-bf7a-447b-9b20-813c2067180f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:20.493254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb4fa92-bf7a-447b-9b20-813c2067180f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:20.511897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1bb4fa92-bf7a-447b-9b20-813c2067180f None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:20.513373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1bb4fa92-bf7a-447b-9b20-813c2067180f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:20.527574 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c23f21-0531-4bfc-9c08-a4dcffad90a8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:20.560915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:20.623756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 198/384] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:20.695725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c23f21-0531-4bfc-9c08-a4dcffad90a8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:20.741921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227da488-7317-4cdd-ba27-8b93a8b8e883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:20.742472 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227da488-7317-4cdd-ba27-8b93a8b8e883 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:20.742747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227da488-7317-4cdd-ba27-8b93a8b8e883 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:20.743015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227da488-7317-4cdd-ba27-8b93a8b8e883 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:20.760700 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227da488-7317-4cdd-ba27-8b93a8b8e883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-227da488-7317-4cdd-ba27-8b93a8b8e883 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:20.765845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:20.766966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:20.767876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:20.767876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:20.767876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:20.767876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:20.767876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:20.767876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:20.767876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:20.767876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:20.767876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:20.767876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:20.767876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:20.767876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:20.767876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 187/385] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:20.773384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4fa92-bf7a-447b-9b20-813c2067180f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:20.842467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:20.912326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb4fa92-bf7a-447b-9b20-813c2067180f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:20.937771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227da488-7317-4cdd-ba27-8b93a8b8e883 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:20.967796 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23eb0b-1f2f-46a7-a31e-5a7985b02ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:20.968478 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23eb0b-1f2f-46a7-a31e-5a7985b02ff6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:20.968828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23eb0b-1f2f-46a7-a31e-5a7985b02ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:20.969074 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23eb0b-1f2f-46a7-a31e-5a7985b02ff6 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:20.969789 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23eb0b-1f2f-46a7-a31e-5a7985b02ff6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:20.970096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23eb0b-1f2f-46a7-a31e-5a7985b02ff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:20.971483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23eb0b-1f2f-46a7-a31e-5a7985b02ff6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:21.007464 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 199/386] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:21.009654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23eb0b-1f2f-46a7-a31e-5a7985b02ff6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:21.054885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:21.078508 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227da488-7317-4cdd-ba27-8b93a8b8e883 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:21.118716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203b61f-821d-4c2f-bb9d-f56bf1c083bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:21.119505 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203b61f-821d-4c2f-bb9d-f56bf1c083bc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:21.119927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203b61f-821d-4c2f-bb9d-f56bf1c083bc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:21.120205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203b61f-821d-4c2f-bb9d-f56bf1c083bc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:21.133421 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203b61f-821d-4c2f-bb9d-f56bf1c083bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c203b61f-821d-4c2f-bb9d-f56bf1c083bc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:21.141062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:21.141952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:21.142750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:21.142750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:21.142750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:21.142750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:21.142750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:21.142750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:21.142750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:21.142750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:21.142750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:21.142750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:21.142750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:21.142750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:21.143334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 188/387] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:44:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:21.150872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203b61f-821d-4c2f-bb9d-f56bf1c083bc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:21.233114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23eb0b-1f2f-46a7-a31e-5a7985b02ff6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:21.258695 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:21.300660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed195e6e-791e-4f78-84ec-5227b3992803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:21.303237 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed195e6e-791e-4f78-84ec-5227b3992803 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:21.303519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed195e6e-791e-4f78-84ec-5227b3992803 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:21.303855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed195e6e-791e-4f78-84ec-5227b3992803 None admin] PATH_INFO: `/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:21.305904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed195e6e-791e-4f78-84ec-5227b3992803 None admin] RBAC: Authorizing `identity:get_group(group_id=6e3090dc484c4fc39d1d9ca6fbdeae29)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:21.307373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed195e6e-791e-4f78-84ec-5227b3992803 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:21.309261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 200/388] 10.222.0.67 () {56 vars in 1180 bytes} [Mon Sep 26 17:44:21 2022] GET /identity/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29 => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:21.318176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed195e6e-791e-4f78-84ec-5227b3992803 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:21.350552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203b61f-821d-4c2f-bb9d-f56bf1c083bc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:21.441781 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60101a3f-9af7-40e6-8770-5d5ceeec9a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:21.442767 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60101a3f-9af7-40e6-8770-5d5ceeec9a22 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:21.443279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60101a3f-9af7-40e6-8770-5d5ceeec9a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:21.443782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60101a3f-9af7-40e6-8770-5d5ceeec9a22 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:21.444763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60101a3f-9af7-40e6-8770-5d5ceeec9a22 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:21.445266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60101a3f-9af7-40e6-8770-5d5ceeec9a22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:21.446856 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60101a3f-9af7-40e6-8770-5d5ceeec9a22 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:21.449168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed195e6e-791e-4f78-84ec-5227b3992803 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:21.457393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 189/389] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:21.463164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b2905-05eb-4161-aaa0-160728c9269a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:21.463766 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b2905-05eb-4161-aaa0-160728c9269a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:21.464265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68b2905-05eb-4161-aaa0-160728c9269a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:21.465771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 190/390] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:21.467051 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b2905-05eb-4161-aaa0-160728c9269a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:21.472341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:21.486365 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bac982-32d8-4423-a7aa-a16a662059f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:21.487004 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bac982-32d8-4423-a7aa-a16a662059f9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:21.487918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bac982-32d8-4423-a7aa-a16a662059f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:21.488199 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bac982-32d8-4423-a7aa-a16a662059f9 None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:21.489412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bac982-32d8-4423-a7aa-a16a662059f9 None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:21.490769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bac982-32d8-4423-a7aa-a16a662059f9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:21.492960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 201/391] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:21 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:21.493940 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b2763-0322-441b-9a27-8bb60b8c35ee None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:21.494271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b2763-0322-441b-9a27-8bb60b8c35ee None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:21.494545 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b2763-0322-441b-9a27-8bb60b8c35ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:21.519212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-804b2763-0322-441b-9a27-8bb60b8c35ee None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:21.519212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-804b2763-0322-441b-9a27-8bb60b8c35ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:21.569435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:21.586366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68b2905-05eb-4161-aaa0-160728c9269a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:21.593445 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 202/392] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:21.594273 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b2763-0322-441b-9a27-8bb60b8c35ee None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:21.625929 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c623f-76c7-447c-bb15-7d9f0a74a67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:21.627046 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c623f-76c7-447c-bb15-7d9f0a74a67e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:21.627560 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c623f-76c7-447c-bb15-7d9f0a74a67e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:21.629000 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c623f-76c7-447c-bb15-7d9f0a74a67e None admin] PATH_INFO: `/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:21.630473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c623f-76c7-447c-bb15-7d9f0a74a67e None admin] RBAC: Authorizing `identity:get_group(group_id=6e3090dc484c4fc39d1d9ca6fbdeae29)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:21.632157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1c623f-76c7-447c-bb15-7d9f0a74a67e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:21.634472 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 191/393] 10.222.0.67 () {56 vars in 1180 bytes} [Mon Sep 26 17:44:21 2022] GET /identity/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:21.640467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c623f-76c7-447c-bb15-7d9f0a74a67e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:21.713436 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b2763-0322-441b-9a27-8bb60b8c35ee None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:21.718322 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:21.759332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c623f-76c7-447c-bb15-7d9f0a74a67e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:21.767715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60f32b-a6d8-44c1-8eef-39b2233690f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:21.768959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60f32b-a6d8-44c1-8eef-39b2233690f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:21.769321 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60f32b-a6d8-44c1-8eef-39b2233690f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:21.769527 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d60f32b-a6d8-44c1-8eef-39b2233690f3 None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01/groups/6e3090dc484c4fc39d1d9ca6fbdeae29/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:21.771674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60f32b-a6d8-44c1-8eef-39b2233690f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=25808aef0f69482dbf49592f737cff01, group_id=6e3090dc484c4fc39d1d9ca6fbdeae29, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:21.773100 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d60f32b-a6d8-44c1-8eef-39b2233690f3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:21.797757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0e104-4f8d-4222-ad79-c19a387b6403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:21.800450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0e104-4f8d-4222-ad79-c19a387b6403 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:21.800450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0e104-4f8d-4222-ad79-c19a387b6403 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:21.800450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0e104-4f8d-4222-ad79-c19a387b6403 None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:21.801793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0e104-4f8d-4222-ad79-c19a387b6403 None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:21.803420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f0e104-4f8d-4222-ad79-c19a387b6403 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:21.805650 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 192/394] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:21 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:21.811400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0e104-4f8d-4222-ad79-c19a387b6403 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:21.863327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 203/395] 10.222.0.67 () {58 vars in 1442 bytes} [Mon Sep 26 17:44:21 2022] PUT /identity/v3/projects/25808aef0f69482dbf49592f737cff01/groups/6e3090dc484c4fc39d1d9ca6fbdeae29/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:21.927003 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f0e104-4f8d-4222-ad79-c19a387b6403 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:21.965591 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14619e6f-8a22-4386-97ad-9687e2ac71ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:21.966763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14619e6f-8a22-4386-97ad-9687e2ac71ad None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:21.967239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14619e6f-8a22-4386-97ad-9687e2ac71ad None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:21.967711 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14619e6f-8a22-4386-97ad-9687e2ac71ad None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01/groups/6e3090dc484c4fc39d1d9ca6fbdeae29/roles/30bb990dfe5c45fbbfed21c08c7859b4` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:21.969633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60f32b-a6d8-44c1-8eef-39b2233690f3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:21.970692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14619e6f-8a22-4386-97ad-9687e2ac71ad None admin] RBAC: Authorizing `identity:create_grant(project_id=25808aef0f69482dbf49592f737cff01, group_id=6e3090dc484c4fc39d1d9ca6fbdeae29, role_id=30bb990dfe5c45fbbfed21c08c7859b4)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:21.973275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14619e6f-8a22-4386-97ad-9687e2ac71ad None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:21.977666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:22.032380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:22.058986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 193/396] 10.222.0.67 () {58 vars in 1442 bytes} [Mon Sep 26 17:44:21 2022] PUT /identity/v3/projects/25808aef0f69482dbf49592f737cff01/groups/6e3090dc484c4fc39d1d9ca6fbdeae29/roles/30bb990dfe5c45fbbfed21c08c7859b4 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 17:44:22.103177 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d60f32b-a6d8-44c1-8eef-39b2233690f3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:22.143956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5a32d-45c3-44bd-a839-1208506e975a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:22.144701 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5a32d-45c3-44bd-a839-1208506e975a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:22.144983 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5a32d-45c3-44bd-a839-1208506e975a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:22.145253 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d5a32d-45c3-44bd-a839-1208506e975a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:22.157865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d5a32d-45c3-44bd-a839-1208506e975a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-58d5a32d-45c3-44bd-a839-1208506e975a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:22.163751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:22.164733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:22.165528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:22.165528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:22.165528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:22.165528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:22.165528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:22.165528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:22.165528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:22.165528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:22.165528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:22.165528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:22.165528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:22.165528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:22.165528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 204/397] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:22.171218 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14619e6f-8a22-4386-97ad-9687e2ac71ad None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:22.305536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14619e6f-8a22-4386-97ad-9687e2ac71ad None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:22.345498 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301d2055-f2cb-490e-97ab-466e0f3bfb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:22.346118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d2055-f2cb-490e-97ab-466e0f3bfb2d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:22.346397 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d2055-f2cb-490e-97ab-466e0f3bfb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:22.347685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301d2055-f2cb-490e-97ab-466e0f3bfb2d None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:22.348368 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d2055-f2cb-490e-97ab-466e0f3bfb2d None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:22.348676 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d2055-f2cb-490e-97ab-466e0f3bfb2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:22.350256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301d2055-f2cb-490e-97ab-466e0f3bfb2d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:22.359363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 194/398] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:22.366012 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5a32d-45c3-44bd-a839-1208506e975a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:22.418369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93190911-5e57-4aa5-a99f-07391c8903cd None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:22.418715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93190911-5e57-4aa5-a99f-07391c8903cd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:22.419632 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93190911-5e57-4aa5-a99f-07391c8903cd None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:22.422283 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 195/399] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:44:22.432057 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4ab02-0d0a-4c2d-8cbd-a06b80f522b8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:22.432403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4ab02-0d0a-4c2d-8cbd-a06b80f522b8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:22.432685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d4ab02-0d0a-4c2d-8cbd-a06b80f522b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:22.461239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-62d4ab02-0d0a-4c2d-8cbd-a06b80f522b8 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:22.462457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-62d4ab02-0d0a-4c2d-8cbd-a06b80f522b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:22.484376 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:22.488509 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d5a32d-45c3-44bd-a839-1208506e975a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:22.525888 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:22.529000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1554468-fc57-41da-aa34-9de0317b5e20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:22.529763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1554468-fc57-41da-aa34-9de0317b5e20 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:22.530141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1554468-fc57-41da-aa34-9de0317b5e20 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:22.530510 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1554468-fc57-41da-aa34-9de0317b5e20 None admin] PATH_INFO: `/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:22.532306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1554468-fc57-41da-aa34-9de0317b5e20 None admin] RBAC: Authorizing `identity:get_group(group_id=6e3090dc484c4fc39d1d9ca6fbdeae29)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:22.538047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1554468-fc57-41da-aa34-9de0317b5e20 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:22.538047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 205/400] 10.222.0.67 () {56 vars in 1180 bytes} [Mon Sep 26 17:44:22 2022] GET /identity/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:22.538047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e722ed-0e30-4eb1-8dac-e39be3df600c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:22.538047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e722ed-0e30-4eb1-8dac-e39be3df600c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:22.538047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e722ed-0e30-4eb1-8dac-e39be3df600c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:22.540318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 206/401] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:22.541055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e722ed-0e30-4eb1-8dac-e39be3df600c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:22.547087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 196/402] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:22.548709 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed84287-6b90-4997-ac5e-2bc986544173 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:22.548995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed84287-6b90-4997-ac5e-2bc986544173 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:22.550694 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed84287-6b90-4997-ac5e-2bc986544173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:22.578288 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2ed84287-6b90-4997-ac5e-2bc986544173 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:22.579404 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2ed84287-6b90-4997-ac5e-2bc986544173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:22.664564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 197/403] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:22.736893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e722ed-0e30-4eb1-8dac-e39be3df600c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:22.774207 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cad9a-e20a-4423-9250-916845560cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:22.774853 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cad9a-e20a-4423-9250-916845560cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:22.775136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cad9a-e20a-4423-9250-916845560cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:22.775411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4cad9a-e20a-4423-9250-916845560cf5 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:22.776596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cad9a-e20a-4423-9250-916845560cf5 None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:22.778040 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4cad9a-e20a-4423-9250-916845560cf5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:22.780064 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 207/404] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:22 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:22.788841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed84287-6b90-4997-ac5e-2bc986544173 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:22.829869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cad9a-e20a-4423-9250-916845560cf5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:22.884057 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:22.957017 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4cad9a-e20a-4423-9250-916845560cf5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:22.998424 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f398986-518e-47ef-bb68-b4700f520b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:22.999118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f398986-518e-47ef-bb68-b4700f520b32 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:22.999404 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f398986-518e-47ef-bb68-b4700f520b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:22.999721 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f398986-518e-47ef-bb68-b4700f520b32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:23.011803 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f398986-518e-47ef-bb68-b4700f520b32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0f398986-518e-47ef-bb68-b4700f520b32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:23.017883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:23.018762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:23.019538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:23.019538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:23.019538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:23.019538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:23.019538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:23.019538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:23.019538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:23.019538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:23.019538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:23.019538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:23.019538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:23.019538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:23.019538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 208/405] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:44:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:44:23.031738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f398986-518e-47ef-bb68-b4700f520b32 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:23.033871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed84287-6b90-4997-ac5e-2bc986544173 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:23.059275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:23.082885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977c456-f1e6-4f8b-87f1-4718de3cb397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:23.084129 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977c456-f1e6-4f8b-87f1-4718de3cb397 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:23.085422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977c456-f1e6-4f8b-87f1-4718de3cb397 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:23.085908 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977c456-f1e6-4f8b-87f1-4718de3cb397 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1/groups/6e3090dc484c4fc39d1d9ca6fbdeae29/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:23.088191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977c456-f1e6-4f8b-87f1-4718de3cb397 None admin] RBAC: Authorizing `identity:create_grant(project_id=887965a382184db9958ad175261f03a1, group_id=6e3090dc484c4fc39d1d9ca6fbdeae29, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:23.090289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0977c456-f1e6-4f8b-87f1-4718de3cb397 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:23.152710 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:23.158955 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f398986-518e-47ef-bb68-b4700f520b32 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:23.195542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8a8af-8dd4-4bcc-b71b-8591eb09f885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:23.195542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8a8af-8dd4-4bcc-b71b-8591eb09f885 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:23.195542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8a8af-8dd4-4bcc-b71b-8591eb09f885 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:23.196055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d8a8af-8dd4-4bcc-b71b-8591eb09f885 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:23.196411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8a8af-8dd4-4bcc-b71b-8591eb09f885 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:23.196703 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8a8af-8dd4-4bcc-b71b-8591eb09f885 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:23.198410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d8a8af-8dd4-4bcc-b71b-8591eb09f885 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:23.202380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 198/406] 10.222.0.67 () {58 vars in 1442 bytes} [Mon Sep 26 17:44:22 2022] PUT /identity/v3/projects/887965a382184db9958ad175261f03a1/groups/6e3090dc484c4fc39d1d9ca6fbdeae29/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:23.208682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977c456-f1e6-4f8b-87f1-4718de3cb397 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:23.225282 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 209/407] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:23.232946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8a8af-8dd4-4bcc-b71b-8591eb09f885 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:23.350107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977c456-f1e6-4f8b-87f1-4718de3cb397 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:23.393310 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fbbac-8a90-48e0-9611-1e3344c51102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:23.394976 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fbbac-8a90-48e0-9611-1e3344c51102 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:23.395462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fbbac-8a90-48e0-9611-1e3344c51102 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:23.395776 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fbbac-8a90-48e0-9611-1e3344c51102 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:23.408559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fbbac-8a90-48e0-9611-1e3344c51102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-866fbbac-8a90-48e0-9611-1e3344c51102 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:23.414827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:23.415755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:23.416550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:23.416550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:23.416550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:23.416550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:23.416550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:23.416550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:23.416550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:23.416550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:23.416550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:23.416550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:23.416550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:23.416550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:23.416550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 199/408] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:44:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:23.424705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fbbac-8a90-48e0-9611-1e3344c51102 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:23.433759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d8a8af-8dd4-4bcc-b71b-8591eb09f885 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:23.478430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675b055-9cb5-4ecf-a05f-3a46d10a58c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:23.479084 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675b055-9cb5-4ecf-a05f-3a46d10a58c9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:23.479364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675b055-9cb5-4ecf-a05f-3a46d10a58c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:23.479660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f675b055-9cb5-4ecf-a05f-3a46d10a58c9 None admin] PATH_INFO: `/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:23.482107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675b055-9cb5-4ecf-a05f-3a46d10a58c9 None admin] RBAC: Authorizing `identity:get_group(group_id=6e3090dc484c4fc39d1d9ca6fbdeae29)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:23.483568 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f675b055-9cb5-4ecf-a05f-3a46d10a58c9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:23.485439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 210/409] 10.222.0.67 () {56 vars in 1180 bytes} [Mon Sep 26 17:44:23 2022] GET /identity/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29 => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:23.492976 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675b055-9cb5-4ecf-a05f-3a46d10a58c9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:23.510398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:23.527172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:23.599546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fbbac-8a90-48e0-9611-1e3344c51102 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:23.607148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f675b055-9cb5-4ecf-a05f-3a46d10a58c9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:23.641727 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085457a3-72e7-4a9c-9dd7-a9343d78ddd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:23.642374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085457a3-72e7-4a9c-9dd7-a9343d78ddd4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:23.642702 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085457a3-72e7-4a9c-9dd7-a9343d78ddd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:23.642978 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085457a3-72e7-4a9c-9dd7-a9343d78ddd4 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:23.644458 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085457a3-72e7-4a9c-9dd7-a9343d78ddd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:23.645002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085457a3-72e7-4a9c-9dd7-a9343d78ddd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:23.646404 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085457a3-72e7-4a9c-9dd7-a9343d78ddd4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:23.657057 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 200/410] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:23.664335 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085457a3-72e7-4a9c-9dd7-a9343d78ddd4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:23.681261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719167a-897e-4605-b4d8-4f59b6deb419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:23.681956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719167a-897e-4605-b4d8-4f59b6deb419 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:23.682235 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719167a-897e-4605-b4d8-4f59b6deb419 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:23.682505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e719167a-897e-4605-b4d8-4f59b6deb419 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:23.683788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719167a-897e-4605-b4d8-4f59b6deb419 None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:23.685305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e719167a-897e-4605-b4d8-4f59b6deb419 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:23.687131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 211/411] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:23 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:23.698928 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719167a-897e-4605-b4d8-4f59b6deb419 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:23.831953 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:23.849821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e719167a-897e-4605-b4d8-4f59b6deb419 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:23.900633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:23.902591 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb45a7-832d-4d19-ab4c-876d890502d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:23.903374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb45a7-832d-4d19-ab4c-876d890502d2 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:23.903797 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb45a7-832d-4d19-ab4c-876d890502d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:23.904182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb45a7-832d-4d19-ab4c-876d890502d2 None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1/groups/6e3090dc484c4fc39d1d9ca6fbdeae29/roles/30bb990dfe5c45fbbfed21c08c7859b4` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:23.904677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085457a3-72e7-4a9c-9dd7-a9343d78ddd4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:23.906799 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb45a7-832d-4d19-ab4c-876d890502d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=887965a382184db9958ad175261f03a1, group_id=6e3090dc484c4fc39d1d9ca6fbdeae29, role_id=30bb990dfe5c45fbbfed21c08c7859b4)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:23.908468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb45a7-832d-4d19-ab4c-876d890502d2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:23.951021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c22b3-7792-4b29-9cf8-dc3e4b592afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:23.951675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c22b3-7792-4b29-9cf8-dc3e4b592afe None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:23.951967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c22b3-7792-4b29-9cf8-dc3e4b592afe None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:23.952240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7c22b3-7792-4b29-9cf8-dc3e4b592afe None admin] PATH_INFO: `/v3/groups/a3b437bb95304f75a3a95b6b679a65cb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:23.953549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c22b3-7792-4b29-9cf8-dc3e4b592afe None admin] RBAC: Authorizing `identity:get_group(group_id=a3b437bb95304f75a3a95b6b679a65cb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:23.954989 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7c22b3-7792-4b29-9cf8-dc3e4b592afe None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:23.957687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 201/412] 10.222.0.67 () {56 vars in 1180 bytes} [Mon Sep 26 17:44:23 2022] GET /identity/v3/groups/a3b437bb95304f75a3a95b6b679a65cb => generated 229 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:23.962949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c22b3-7792-4b29-9cf8-dc3e4b592afe None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:24.026195 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 212/413] 10.222.0.67 () {58 vars in 1442 bytes} [Mon Sep 26 17:44:23 2022] PUT /identity/v3/projects/887965a382184db9958ad175261f03a1/groups/6e3090dc484c4fc39d1d9ca6fbdeae29/roles/30bb990dfe5c45fbbfed21c08c7859b4 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:24.093622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7c22b3-7792-4b29-9cf8-dc3e4b592afe None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:24.149090 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0abf67-0eb4-4ea7-8a90-c146a9b34d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:24.150105 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0abf67-0eb4-4ea7-8a90-c146a9b34d63 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:24.150564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0abf67-0eb4-4ea7-8a90-c146a9b34d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:24.151071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0abf67-0eb4-4ea7-8a90-c146a9b34d63 None admin] PATH_INFO: `/v3/projects/1537d57f83734659b1130d2661d44ac2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:24.152530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0abf67-0eb4-4ea7-8a90-c146a9b34d63 None admin] RBAC: Authorizing `identity:get_project(project_id=1537d57f83734659b1130d2661d44ac2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:24.154227 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0abf67-0eb4-4ea7-8a90-c146a9b34d63 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:24.156473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 202/414] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:23 2022] GET /identity/v3/projects/1537d57f83734659b1130d2661d44ac2 => generated 344 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:24.163426 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb45a7-832d-4d19-ab4c-876d890502d2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:24.319689 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb45a7-832d-4d19-ab4c-876d890502d2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:24.354949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc9eac6-3489-4608-966f-519dcb87504c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:24.355631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9eac6-3489-4608-966f-519dcb87504c None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:24.355943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9eac6-3489-4608-966f-519dcb87504c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:24.356227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc9eac6-3489-4608-966f-519dcb87504c None admin] PATH_INFO: `/v3/projects/1537d57f83734659b1130d2661d44ac2/groups/a3b437bb95304f75a3a95b6b679a65cb/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:24.358159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9eac6-3489-4608-966f-519dcb87504c None admin] RBAC: Authorizing `identity:create_grant(project_id=1537d57f83734659b1130d2661d44ac2, group_id=a3b437bb95304f75a3a95b6b679a65cb, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:24.359595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc9eac6-3489-4608-966f-519dcb87504c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:24.499254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 213/415] 10.222.0.67 () {58 vars in 1442 bytes} [Mon Sep 26 17:44:24 2022] PUT /identity/v3/projects/1537d57f83734659b1130d2661d44ac2/groups/a3b437bb95304f75a3a95b6b679a65cb/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:24.521391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd72a0d-8e74-44cb-9069-c3be3d4d9b64 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:24.521704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd72a0d-8e74-44cb-9069-c3be3d4d9b64 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:24.521939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd72a0d-8e74-44cb-9069-c3be3d4d9b64 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:24.523396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 203/416] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:24.531703 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99812357-d3be-4282-8544-9108e0677561 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:24.531703 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99812357-d3be-4282-8544-9108e0677561 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:24.531703 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99812357-d3be-4282-8544-9108e0677561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:24.558455 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-99812357-d3be-4282-8544-9108e0677561 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:24.560226 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-99812357-d3be-4282-8544-9108e0677561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:24.577954 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:24.657915 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 214/417] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:24.854485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a4b6b-20ed-40fb-b25e-6cc0e2c18166 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:24.854779 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a4b6b-20ed-40fb-b25e-6cc0e2c18166 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:24.854968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a4b6b-20ed-40fb-b25e-6cc0e2c18166 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:24.856323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 204/418] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:24.861753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591cb2d-2f76-492f-8692-e2c5eb7f04b8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:24.862044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591cb2d-2f76-492f-8692-e2c5eb7f04b8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:24.862360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6591cb2d-2f76-492f-8692-e2c5eb7f04b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:24.882526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6591cb2d-2f76-492f-8692-e2c5eb7f04b8 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:24.883745 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6591cb2d-2f76-492f-8692-e2c5eb7f04b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:24.971425 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 215/419] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:25.004897 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a4b6b-20ed-40fb-b25e-6cc0e2c18166 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:25.064694 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:25.155032 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a4b6b-20ed-40fb-b25e-6cc0e2c18166 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:25.202796 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53bdc6-c069-4daf-b403-2b9ce915bad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:25.203437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53bdc6-c069-4daf-b403-2b9ce915bad8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:25.203748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53bdc6-c069-4daf-b403-2b9ce915bad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:25.204072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb53bdc6-c069-4daf-b403-2b9ce915bad8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:25.224868 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb53bdc6-c069-4daf-b403-2b9ce915bad8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-bb53bdc6-c069-4daf-b403-2b9ce915bad8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:25.230385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:25.231259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:25.232100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:25.232100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:25.232100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:25.232100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:25.232100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:25.232100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:25.232100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:25.232100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:25.232100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:25.232100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:25.232100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:25.232100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:25.232100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 205/420] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:25.238721 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591cb2d-2f76-492f-8692-e2c5eb7f04b8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:25.296964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53bdc6-c069-4daf-b403-2b9ce915bad8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:25.368025 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:25.422441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6591cb2d-2f76-492f-8692-e2c5eb7f04b8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:25.430269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:25.437700 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb53bdc6-c069-4daf-b403-2b9ce915bad8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:25.479869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2077de-3ecb-45df-807a-2973e3633961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:25.481216 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2077de-3ecb-45df-807a-2973e3633961 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:25.481784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2077de-3ecb-45df-807a-2973e3633961 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:25.482353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2077de-3ecb-45df-807a-2973e3633961 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:25.495199 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2077de-3ecb-45df-807a-2973e3633961 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3b2077de-3ecb-45df-807a-2973e3633961 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:25.500485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:25.501381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:25.502178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:25.502178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:25.502178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:25.502178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:25.502178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:25.502178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:25.502178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:25.502178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:25.502178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:25.502178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:25.502178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:25.502178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:25.504354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f526325-4756-4817-a0d6-9d6c404c8766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:25.504948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 206/421] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:44:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:25.505273 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f526325-4756-4817-a0d6-9d6c404c8766 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:25.505558 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f526325-4756-4817-a0d6-9d6c404c8766 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:25.505827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f526325-4756-4817-a0d6-9d6c404c8766 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:25.506569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f526325-4756-4817-a0d6-9d6c404c8766 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:25.506869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f526325-4756-4817-a0d6-9d6c404c8766 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:25.509748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f526325-4756-4817-a0d6-9d6c404c8766 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:25.510722 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2077de-3ecb-45df-807a-2973e3633961 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:25.526834 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 216/422] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:25.532599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f526325-4756-4817-a0d6-9d6c404c8766 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:25.697951 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f526325-4756-4817-a0d6-9d6c404c8766 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:25.737017 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b27df4-9516-4689-842e-94d53612b2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:25.738338 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b27df4-9516-4689-842e-94d53612b2f3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:25.738624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b27df4-9516-4689-842e-94d53612b2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:25.738898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b27df4-9516-4689-842e-94d53612b2f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:25.741228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2077de-3ecb-45df-807a-2973e3633961 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:25.750372 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:25.827680 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b27df4-9516-4689-842e-94d53612b2f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:25.827680 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b27df4-9516-4689-842e-94d53612b2f3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:25.829913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb9c31-e26d-4e6a-a61c-5a2944e91f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:25.831441 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb9c31-e26d-4e6a-a61c-5a2944e91f26 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:25.831988 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb9c31-e26d-4e6a-a61c-5a2944e91f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:25.832436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb9c31-e26d-4e6a-a61c-5a2944e91f26 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:25.833320 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb9c31-e26d-4e6a-a61c-5a2944e91f26 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:25.833829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb9c31-e26d-4e6a-a61c-5a2944e91f26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:25.835523 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb9c31-e26d-4e6a-a61c-5a2944e91f26 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:25.859043 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 207/423] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:25.865220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb9c31-e26d-4e6a-a61c-5a2944e91f26 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:25.903107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:26.001211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 217/424] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 17:44:26.002600 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac3f83-ba99-4603-95df-e36fef3a4794 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:26.002759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb9c31-e26d-4e6a-a61c-5a2944e91f26 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:26.003628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac3f83-ba99-4603-95df-e36fef3a4794 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:26.003994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac3f83-ba99-4603-95df-e36fef3a4794 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:26.005337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 218/425] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:26.008867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac3f83-ba99-4603-95df-e36fef3a4794 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:26.046830 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45099a54-1426-4a09-88f3-b686da1e1b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:26.048226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45099a54-1426-4a09-88f3-b686da1e1b77 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:26.048702 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45099a54-1426-4a09-88f3-b686da1e1b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:26.049196 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45099a54-1426-4a09-88f3-b686da1e1b77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:26.054707 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:26.142122 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac3f83-ba99-4603-95df-e36fef3a4794 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:26.148527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45099a54-1426-4a09-88f3-b686da1e1b77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:26.150062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45099a54-1426-4a09-88f3-b686da1e1b77 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:26.170582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:26.185409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f83662-d051-42b9-b297-3dae8f0c285f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:26.190902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f83662-d051-42b9-b297-3dae8f0c285f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:26.190902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f83662-d051-42b9-b297-3dae8f0c285f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:26.190902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f83662-d051-42b9-b297-3dae8f0c285f None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:26.190902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f83662-d051-42b9-b297-3dae8f0c285f None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:26.190902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f83662-d051-42b9-b297-3dae8f0c285f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:26.195783 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 219/426] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:26 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:26.197078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b9db3-1c6e-4e75-8d18-99f1f1d3b368 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:26.197370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b9db3-1c6e-4e75-8d18-99f1f1d3b368 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:26.197648 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2b9db3-1c6e-4e75-8d18-99f1f1d3b368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:26.219818 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-eb2b9db3-1c6e-4e75-8d18-99f1f1d3b368 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:26.221099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-eb2b9db3-1c6e-4e75-8d18-99f1f1d3b368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:26.260610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 208/427] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 17:44:26.261753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45099a54-1426-4a09-88f3-b686da1e1b77 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:26.314712 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 220/428] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:26.316060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b9db3-1c6e-4e75-8d18-99f1f1d3b368 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:26.360582 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:26.369566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45099a54-1426-4a09-88f3-b686da1e1b77 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:26.389895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:26.408752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a85b8-0427-4fa2-b965-6611741ab6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:26.409802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a85b8-0427-4fa2-b965-6611741ab6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:26.410310 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a85b8-0427-4fa2-b965-6611741ab6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:26.410886 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193a85b8-0427-4fa2-b965-6611741ab6f7 None admin] PATH_INFO: `/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:26.412229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a85b8-0427-4fa2-b965-6611741ab6f7 None admin] RBAC: Authorizing `identity:get_group(group_id=6e3090dc484c4fc39d1d9ca6fbdeae29)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:26.414246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193a85b8-0427-4fa2-b965-6611741ab6f7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:26.416403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 209/429] 10.222.0.67 () {56 vars in 1180 bytes} [Mon Sep 26 17:44:26 2022] GET /identity/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29 => generated 226 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:26.426905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a85b8-0427-4fa2-b965-6611741ab6f7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:26.464804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2b9db3-1c6e-4e75-8d18-99f1f1d3b368 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:26.507055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3362d3e-5950-4d8c-9572-1e45e1339df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:26.507720 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3362d3e-5950-4d8c-9572-1e45e1339df6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:26.508001 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3362d3e-5950-4d8c-9572-1e45e1339df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:26.508271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3362d3e-5950-4d8c-9572-1e45e1339df6 None admin] PATH_INFO: `/v3/projects/25808aef0f69482dbf49592f737cff01` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:26.512189 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3362d3e-5950-4d8c-9572-1e45e1339df6 None admin] RBAC: Authorizing `identity:get_project(project_id=25808aef0f69482dbf49592f737cff01)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:26.515391 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3362d3e-5950-4d8c-9572-1e45e1339df6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:26.518949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 221/430] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:26 2022] GET /identity/v3/projects/25808aef0f69482dbf49592f737cff01 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:26.523004 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3362d3e-5950-4d8c-9572-1e45e1339df6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:26.629702 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193a85b8-0427-4fa2-b965-6611741ab6f7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:26.690079 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7bff2-01de-48ec-a346-8f3aa346080c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:26.691796 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7bff2-01de-48ec-a346-8f3aa346080c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:26.692299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7bff2-01de-48ec-a346-8f3aa346080c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:26.692769 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f7bff2-01de-48ec-a346-8f3aa346080c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:26.693735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7bff2-01de-48ec-a346-8f3aa346080c None admin] RBAC: Adding query filter params (group.id=6e3090dc484c4fc39d1d9ca6fbdeae29, role.id=1c983c7e32e44c61be3996c28399e617, scope.project.id=25808aef0f69482dbf49592f737cff01) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:26.694250 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7bff2-01de-48ec-a346-8f3aa346080c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:26.694698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3362d3e-5950-4d8c-9572-1e45e1339df6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:26.696242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f7bff2-01de-48ec-a346-8f3aa346080c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:26.696506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:26.704011 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 210/431] 10.222.0.67 () {56 vars in 1376 bytes} [Mon Sep 26 17:44:26 2022] GET /identity/v3/role_assignments?group.id=6e3090dc484c4fc39d1d9ca6fbdeae29&scope.project.id=25808aef0f69482dbf49592f737cff01&role.id=1c983c7e32e44c61be3996c28399e617 => generated 611 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:26.705287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7bff2-01de-48ec-a346-8f3aa346080c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:26.744475 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f860b92-4dc9-43ea-a1a8-b3f2006d5594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:26.745135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f860b92-4dc9-43ea-a1a8-b3f2006d5594 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:26.745375 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f860b92-4dc9-43ea-a1a8-b3f2006d5594 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:26.745647 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f860b92-4dc9-43ea-a1a8-b3f2006d5594 None admin] PATH_INFO: `/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:26.748419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f860b92-4dc9-43ea-a1a8-b3f2006d5594 None admin] RBAC: Authorizing `identity:get_group(group_id=6e3090dc484c4fc39d1d9ca6fbdeae29)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:26.748419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f860b92-4dc9-43ea-a1a8-b3f2006d5594 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:26.750623 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 222/432] 10.222.0.67 () {56 vars in 1180 bytes} [Mon Sep 26 17:44:26 2022] GET /identity/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:26.756880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f860b92-4dc9-43ea-a1a8-b3f2006d5594 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:26.812748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:26.879860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f7bff2-01de-48ec-a346-8f3aa346080c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:26.916841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f860b92-4dc9-43ea-a1a8-b3f2006d5594 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:26.921961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb72a7-b9b6-4ddf-879a-fae1d162c725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:26.922597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb72a7-b9b6-4ddf-879a-fae1d162c725 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:26.922873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb72a7-b9b6-4ddf-879a-fae1d162c725 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:26.923141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb72a7-b9b6-4ddf-879a-fae1d162c725 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:26.939417 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb72a7-b9b6-4ddf-879a-fae1d162c725 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-48eb72a7-b9b6-4ddf-879a-fae1d162c725 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:26.948055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:26.948984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:26.949747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:26.949747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:26.949747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:26.949747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:26.949747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:26.949747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:26.949747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:26.949747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:26.949747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:26.949747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:26.949747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 26 17:44:26.949747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:26.950329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 211/433] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:26.951126 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc76f7d-7352-4289-b30d-ca05767755d8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:26.951411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc76f7d-7352-4289-b30d-ca05767755d8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:26.951710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc76f7d-7352-4289-b30d-ca05767755d8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:26.952957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 212/434] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:26.957484 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc76f7d-7352-4289-b30d-ca05767755d8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:26.962232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dadb790-5b0d-43a0-acbc-94d6c006f686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:26.962856 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dadb790-5b0d-43a0-acbc-94d6c006f686 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:26.963129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dadb790-5b0d-43a0-acbc-94d6c006f686 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:26.963395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dadb790-5b0d-43a0-acbc-94d6c006f686 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:26.964157 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dadb790-5b0d-43a0-acbc-94d6c006f686 None admin] RBAC: Adding query filter params (group.id=6e3090dc484c4fc39d1d9ca6fbdeae29, role.id=30bb990dfe5c45fbbfed21c08c7859b4, scope.project.id=25808aef0f69482dbf49592f737cff01) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:26.964466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dadb790-5b0d-43a0-acbc-94d6c006f686 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:26.965904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dadb790-5b0d-43a0-acbc-94d6c006f686 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:26.973450 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 223/435] 10.222.0.67 () {56 vars in 1376 bytes} [Mon Sep 26 17:44:26 2022] GET /identity/v3/role_assignments?group.id=6e3090dc484c4fc39d1d9ca6fbdeae29&scope.project.id=25808aef0f69482dbf49592f737cff01&role.id=30bb990dfe5c45fbbfed21c08c7859b4 => generated 611 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:26.974800 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aea7b-b8a4-43b3-aa1f-289855b41f67 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:26.975082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aea7b-b8a4-43b3-aa1f-289855b41f67 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:26.976190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12aea7b-b8a4-43b3-aa1f-289855b41f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:26.998445 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a12aea7b-b8a4-43b3-aa1f-289855b41f67 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:26.999100 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a12aea7b-b8a4-43b3-aa1f-289855b41f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:27.040057 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:27.098080 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc76f7d-7352-4289-b30d-ca05767755d8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:27.099248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 224/436] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:27.115433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:27.142230 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03e5ad6-4ca0-4b7e-b39c-3374cba80ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:27.142849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e5ad6-4ca0-4b7e-b39c-3374cba80ccb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:27.143127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e5ad6-4ca0-4b7e-b39c-3374cba80ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:27.143421 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03e5ad6-4ca0-4b7e-b39c-3374cba80ccb None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:27.144250 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e5ad6-4ca0-4b7e-b39c-3374cba80ccb None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:27.144638 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e5ad6-4ca0-4b7e-b39c-3374cba80ccb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:27.146052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03e5ad6-4ca0-4b7e-b39c-3374cba80ccb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:27.157154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 213/437] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:44:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:27.163617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12aea7b-b8a4-43b3-aa1f-289855b41f67 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:27.293746 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12aea7b-b8a4-43b3-aa1f-289855b41f67 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:27.331457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9085ffb-d2e3-4987-a32f-8e1e1b3b1dbc None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:27.331829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9085ffb-d2e3-4987-a32f-8e1e1b3b1dbc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:27.332027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9085ffb-d2e3-4987-a32f-8e1e1b3b1dbc None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:27.333536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 214/438] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:27.334784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2ff9a-682e-426c-8acc-b57b2cb01baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:27.335354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2ff9a-682e-426c-8acc-b57b2cb01baa None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:27.335701 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2ff9a-682e-426c-8acc-b57b2cb01baa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:27.335946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c2ff9a-682e-426c-8acc-b57b2cb01baa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:27.340017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b8143-1e40-4716-8b33-d373ed70d62d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:27.340369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b8143-1e40-4716-8b33-d373ed70d62d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:27.340737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7b8143-1e40-4716-8b33-d373ed70d62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:27.362565 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-eb7b8143-1e40-4716-8b33-d373ed70d62d None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:27.363983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eb7b8143-1e40-4716-8b33-d373ed70d62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:27.374373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:27.419454 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2ff9a-682e-426c-8acc-b57b2cb01baa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:27.421762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c2ff9a-682e-426c-8acc-b57b2cb01baa None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:27.458619 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 215/439] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:27.459305 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b8143-1e40-4716-8b33-d373ed70d62d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:27.551152 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 225/440] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:27.557299 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2ff9a-682e-426c-8acc-b57b2cb01baa None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:27.586620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7b8143-1e40-4716-8b33-d373ed70d62d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:27.588362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:27.635997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9991b08-1e56-46f4-ad06-3812704d35f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:27.636637 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9991b08-1e56-46f4-ad06-3812704d35f6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:27.636916 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9991b08-1e56-46f4-ad06-3812704d35f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:27.637186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9991b08-1e56-46f4-ad06-3812704d35f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:27.642025 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:27.649045 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9991b08-1e56-46f4-ad06-3812704d35f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f9991b08-1e56-46f4-ad06-3812704d35f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:27.654244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:27.655098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:27.660475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:27.660475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:27.660475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:27.660475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:27.660475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:27.660475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:27.660475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:27.660475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:27.660475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:27.660475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:27.660475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:27.660475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:27.660475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 216/441] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:44:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:27.661500 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9991b08-1e56-46f4-ad06-3812704d35f6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:27.686926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c2ff9a-682e-426c-8acc-b57b2cb01baa None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:27.725019 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea2075-93c2-40d3-8493-ca785882c83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:27.725660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea2075-93c2-40d3-8493-ca785882c83f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:27.726195 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea2075-93c2-40d3-8493-ca785882c83f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:27.726195 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dea2075-93c2-40d3-8493-ca785882c83f None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:27.727318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea2075-93c2-40d3-8493-ca785882c83f None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:27.728804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dea2075-93c2-40d3-8493-ca785882c83f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:27.730649 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 226/442] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:27 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:27.736099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea2075-93c2-40d3-8493-ca785882c83f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:27.766776 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9991b08-1e56-46f4-ad06-3812704d35f6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:27.800415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d1f0e-b688-403c-bb36-0a72c45a5d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:27.801021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d1f0e-b688-403c-bb36-0a72c45a5d9b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:27.801297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d1f0e-b688-403c-bb36-0a72c45a5d9b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:27.801565 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d1f0e-b688-403c-bb36-0a72c45a5d9b None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:27.802224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d1f0e-b688-403c-bb36-0a72c45a5d9b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:27.802520 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d1f0e-b688-403c-bb36-0a72c45a5d9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:27.803911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66d1f0e-b688-403c-bb36-0a72c45a5d9b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:27.811861 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 217/443] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:27.816476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d1f0e-b688-403c-bb36-0a72c45a5d9b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:27.836482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dea2075-93c2-40d3-8493-ca785882c83f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:27.841022 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:27.865929 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:27.871170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ab78f-c7a9-4b20-a0c3-b78a5469fdb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:27.871845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ab78f-c7a9-4b20-a0c3-b78a5469fdb7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:27.872129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ab78f-c7a9-4b20-a0c3-b78a5469fdb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:27.872400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4ab78f-c7a9-4b20-a0c3-b78a5469fdb7 None admin] PATH_INFO: `/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:27.874078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ab78f-c7a9-4b20-a0c3-b78a5469fdb7 None admin] RBAC: Authorizing `identity:get_group(group_id=6e3090dc484c4fc39d1d9ca6fbdeae29)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:27.876872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4ab78f-c7a9-4b20-a0c3-b78a5469fdb7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:27.885171 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 227/444] 10.222.0.67 () {56 vars in 1180 bytes} [Mon Sep 26 17:44:27 2022] GET /identity/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29 => generated 226 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:27.885171 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ab78f-c7a9-4b20-a0c3-b78a5469fdb7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:27.917764 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d1f0e-b688-403c-bb36-0a72c45a5d9b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:27.951133 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b561e4-bcb6-41e1-831f-eeebfefc46d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:27.951779 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b561e4-bcb6-41e1-831f-eeebfefc46d0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:27.952060 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b561e4-bcb6-41e1-831f-eeebfefc46d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:27.952329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b561e4-bcb6-41e1-831f-eeebfefc46d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:27.962719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b561e4-bcb6-41e1-831f-eeebfefc46d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c5b561e4-bcb6-41e1-831f-eeebfefc46d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:27.968579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:27.969434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:27.970513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:27.970513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:27.970513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:27.970513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:27.970513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:27.970513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:27.970513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:27.970513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:27.970513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:27.970513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:27.970513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:27.970513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:27.970513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 218/445] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:44:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:27.971401 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b561e4-bcb6-41e1-831f-eeebfefc46d0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:27.987593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4ab78f-c7a9-4b20-a0c3-b78a5469fdb7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:28.023173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aba104-2399-423c-8722-9d7e8016f1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:28.023810 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aba104-2399-423c-8722-9d7e8016f1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:28.024089 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aba104-2399-423c-8722-9d7e8016f1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:28.024360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33aba104-2399-423c-8722-9d7e8016f1b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:28.057080 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:28.072499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b561e4-bcb6-41e1-831f-eeebfefc46d0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:28.088843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:28.091049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aba104-2399-423c-8722-9d7e8016f1b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:28.092527 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33aba104-2399-423c-8722-9d7e8016f1b9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:28.107445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7eb118-f617-443c-82b6-b8df5d181b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:28.108091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7eb118-f617-443c-82b6-b8df5d181b68 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:28.108367 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7eb118-f617-443c-82b6-b8df5d181b68 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:28.108640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7eb118-f617-443c-82b6-b8df5d181b68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:28.109370 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7eb118-f617-443c-82b6-b8df5d181b68 None admin] RBAC: Adding query filter params (group.id=6e3090dc484c4fc39d1d9ca6fbdeae29, role.id=1c983c7e32e44c61be3996c28399e617, scope.project.id=887965a382184db9958ad175261f03a1) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:28.109682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7eb118-f617-443c-82b6-b8df5d181b68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:28.111115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7eb118-f617-443c-82b6-b8df5d181b68 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:28.131691 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 219/446] 10.222.0.67 () {56 vars in 1376 bytes} [Mon Sep 26 17:44:27 2022] GET /identity/v3/role_assignments?group.id=6e3090dc484c4fc39d1d9ca6fbdeae29&scope.project.id=887965a382184db9958ad175261f03a1&role.id=1c983c7e32e44c61be3996c28399e617 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:28.131691 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7eb118-f617-443c-82b6-b8df5d181b68 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:28.206846 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 228/447] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:28.212521 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aba104-2399-423c-8722-9d7e8016f1b9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:28.228960 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7eb118-f617-443c-82b6-b8df5d181b68 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:28.272438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fca37-0651-4925-853c-10b9d1af0467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:28.273356 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fca37-0651-4925-853c-10b9d1af0467 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:28.273356 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fca37-0651-4925-853c-10b9d1af0467 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:28.273604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fca37-0651-4925-853c-10b9d1af0467 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:28.274377 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53fca37-0651-4925-853c-10b9d1af0467 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:28.274586 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53fca37-0651-4925-853c-10b9d1af0467 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:28.276009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53fca37-0651-4925-853c-10b9d1af0467 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:28.286582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 220/448] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:28.292148 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fca37-0651-4925-853c-10b9d1af0467 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:28.307048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:28.329260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:28.334581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33aba104-2399-423c-8722-9d7e8016f1b9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:28.372636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3839a-af77-424f-86fa-ff9107241f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:28.373279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3839a-af77-424f-86fa-ff9107241f4e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:28.373564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3839a-af77-424f-86fa-ff9107241f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:28.373840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d3839a-af77-424f-86fa-ff9107241f4e None admin] PATH_INFO: `/v3/projects/887965a382184db9958ad175261f03a1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:28.375496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3839a-af77-424f-86fa-ff9107241f4e None admin] RBAC: Authorizing `identity:get_project(project_id=887965a382184db9958ad175261f03a1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:28.376419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d3839a-af77-424f-86fa-ff9107241f4e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:28.378764 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 229/449] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:28 2022] GET /identity/v3/projects/887965a382184db9958ad175261f03a1 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:28.384677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3839a-af77-424f-86fa-ff9107241f4e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:28.420332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fca37-0651-4925-853c-10b9d1af0467 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:28.462110 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925baed2-fb42-4163-a651-7d11c66803da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:28.462654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925baed2-fb42-4163-a651-7d11c66803da None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:28.462936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925baed2-fb42-4163-a651-7d11c66803da None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:28.463217 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925baed2-fb42-4163-a651-7d11c66803da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:28.501517 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d3839a-af77-424f-86fa-ff9107241f4e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:28.539106 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab2cdc-526f-4405-bc32-09d6b8a21708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:28.539775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab2cdc-526f-4405-bc32-09d6b8a21708 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:28.540056 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab2cdc-526f-4405-bc32-09d6b8a21708 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:28.540327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab2cdc-526f-4405-bc32-09d6b8a21708 None admin] PATH_INFO: `/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:28.541479 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab2cdc-526f-4405-bc32-09d6b8a21708 None admin] RBAC: Authorizing `identity:get_group(group_id=6e3090dc484c4fc39d1d9ca6fbdeae29)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:28.541819 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925baed2-fb42-4163-a651-7d11c66803da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:28.542914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab2cdc-526f-4405-bc32-09d6b8a21708 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:28.543253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925baed2-fb42-4163-a651-7d11c66803da None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:28.544490 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 230/450] 10.222.0.67 () {56 vars in 1180 bytes} [Mon Sep 26 17:44:28 2022] GET /identity/v3/groups/6e3090dc484c4fc39d1d9ca6fbdeae29 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:28.550316 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab2cdc-526f-4405-bc32-09d6b8a21708 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:28.555106 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:28.585557 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:28.645388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 221/451] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 17:44:28.650921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925baed2-fb42-4163-a651-7d11c66803da None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:28.652113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab2cdc-526f-4405-bc32-09d6b8a21708 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:28.685995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a020e8-7f90-4dc0-bc41-f8106412d637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:28.686606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a020e8-7f90-4dc0-bc41-f8106412d637 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:28.686886 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a020e8-7f90-4dc0-bc41-f8106412d637 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:28.687157 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a020e8-7f90-4dc0-bc41-f8106412d637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:28.687903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a020e8-7f90-4dc0-bc41-f8106412d637 None admin] RBAC: Adding query filter params (group.id=6e3090dc484c4fc39d1d9ca6fbdeae29, role.id=30bb990dfe5c45fbbfed21c08c7859b4, scope.project.id=887965a382184db9958ad175261f03a1) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:28.688212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a020e8-7f90-4dc0-bc41-f8106412d637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:28.689633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a020e8-7f90-4dc0-bc41-f8106412d637 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:28.696106 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 231/452] 10.222.0.67 () {56 vars in 1376 bytes} [Mon Sep 26 17:44:28 2022] GET /identity/v3/role_assignments?group.id=6e3090dc484c4fc39d1d9ca6fbdeae29&scope.project.id=887965a382184db9958ad175261f03a1&role.id=30bb990dfe5c45fbbfed21c08c7859b4 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:28.761928 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925baed2-fb42-4163-a651-7d11c66803da None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:28.775182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:28.800425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84dae9-3260-4487-acb0-f2f1dceea07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:28.801255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84dae9-3260-4487-acb0-f2f1dceea07e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:28.801543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84dae9-3260-4487-acb0-f2f1dceea07e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:28.801822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab84dae9-3260-4487-acb0-f2f1dceea07e None admin] PATH_INFO: `/v3/projects/1537d57f83734659b1130d2661d44ac2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:28.803072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84dae9-3260-4487-acb0-f2f1dceea07e None admin] RBAC: Authorizing `identity:get_project(project_id=1537d57f83734659b1130d2661d44ac2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:28.804656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab84dae9-3260-4487-acb0-f2f1dceea07e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:28.806319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 222/453] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:44:28 2022] GET /identity/v3/projects/1537d57f83734659b1130d2661d44ac2 => generated 344 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:28.812057 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a020e8-7f90-4dc0-bc41-f8106412d637 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:28.915145 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:28.918043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a020e8-7f90-4dc0-bc41-f8106412d637 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:28.951362 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f0b0e1-7358-4a76-bfd1-2d44bd39b48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:28.952032 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0b0e1-7358-4a76-bfd1-2d44bd39b48c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:28.952310 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0b0e1-7358-4a76-bfd1-2d44bd39b48c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:28.952587 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f0b0e1-7358-4a76-bfd1-2d44bd39b48c None admin] PATH_INFO: `/v3/groups/a3b437bb95304f75a3a95b6b679a65cb` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:28.953683 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0b0e1-7358-4a76-bfd1-2d44bd39b48c None admin] RBAC: Authorizing `identity:get_group(group_id=a3b437bb95304f75a3a95b6b679a65cb)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:28.955160 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f0b0e1-7358-4a76-bfd1-2d44bd39b48c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:28.957915 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 232/454] 10.222.0.67 () {56 vars in 1180 bytes} [Mon Sep 26 17:44:28 2022] GET /identity/v3/groups/a3b437bb95304f75a3a95b6b679a65cb => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:28.963754 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84dae9-3260-4487-acb0-f2f1dceea07e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:29.069322 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab84dae9-3260-4487-acb0-f2f1dceea07e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:29.102940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e5b26-cc95-41c4-8fc3-331ac7c2cc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:29.103538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e5b26-cc95-41c4-8fc3-331ac7c2cc02 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:29.103856 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e5b26-cc95-41c4-8fc3-331ac7c2cc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:29.104129 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e5b26-cc95-41c4-8fc3-331ac7c2cc02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:29.104848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e5b26-cc95-41c4-8fc3-331ac7c2cc02 None admin] RBAC: Adding query filter params (group.id=a3b437bb95304f75a3a95b6b679a65cb, role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=1537d57f83734659b1130d2661d44ac2) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:29.105159 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e5b26-cc95-41c4-8fc3-331ac7c2cc02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:29.106584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e5b26-cc95-41c4-8fc3-331ac7c2cc02 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:29.112922 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 223/455] 10.222.0.67 () {56 vars in 1376 bytes} [Mon Sep 26 17:44:28 2022] GET /identity/v3/role_assignments?group.id=a3b437bb95304f75a3a95b6b679a65cb&scope.project.id=1537d57f83734659b1130d2661d44ac2&role.id=07626899472b4e32adc0c71ce796a25d => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:31.670760 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6aaa6a-ba38-4b50-9326-9683caf7e056 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:31.671803 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6aaa6a-ba38-4b50-9326-9683caf7e056 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:31.672315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6aaa6a-ba38-4b50-9326-9683caf7e056 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:31.674133 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 233/456] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:31.680001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b7dff-9938-486c-8091-4876e3a3eb59 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:31.680434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b7dff-9938-486c-8091-4876e3a3eb59 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:31.680535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b7dff-9938-486c-8091-4876e3a3eb59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:31.703956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b40b7dff-9938-486c-8091-4876e3a3eb59 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:31.705410 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b40b7dff-9938-486c-8091-4876e3a3eb59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:31.755163 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:31.795108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 224/457] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:32.072735 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32c736-1397-41be-b356-0d14a15b02e7 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:32.073366 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32c736-1397-41be-b356-0d14a15b02e7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:32.073855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32c736-1397-41be-b356-0d14a15b02e7 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:32.075373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 234/458] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:32.080755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9e814-d89f-4071-9ec8-f3baaad9f579 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:32.081108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9e814-d89f-4071-9ec8-f3baaad9f579 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:32.081415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9e814-d89f-4071-9ec8-f3baaad9f579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:32.105227 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-abe9e814-d89f-4071-9ec8-f3baaad9f579 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:32.105433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32c736-1397-41be-b356-0d14a15b02e7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:32.106424 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-abe9e814-d89f-4071-9ec8-f3baaad9f579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:32.204767 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 225/459] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:32.207362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622a6d3-8a27-4182-9ab4-1c0cf80905c4 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:32.207916 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622a6d3-8a27-4182-9ab4-1c0cf80905c4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:32.208396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622a6d3-8a27-4182-9ab4-1c0cf80905c4 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:32.209917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 226/460] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:32.221101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4de1b-fb9a-4a85-9f81-2053b7397f59 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:32.222357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4de1b-fb9a-4a85-9f81-2053b7397f59 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:32.222836 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f4de1b-fb9a-4a85-9f81-2053b7397f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:32.255735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f8f4de1b-fb9a-4a85-9f81-2053b7397f59 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:32.258680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f8f4de1b-fb9a-4a85-9f81-2053b7397f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:32.297491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32c736-1397-41be-b356-0d14a15b02e7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:32.383102 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 227/461] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:32.385001 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6971918-17a7-4a90-91c8-af81b32cce0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:32.385903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6971918-17a7-4a90-91c8-af81b32cce0a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:32.386436 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6971918-17a7-4a90-91c8-af81b32cce0a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:32.386919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6971918-17a7-4a90-91c8-af81b32cce0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:32.401254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6971918-17a7-4a90-91c8-af81b32cce0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:32.403210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6971918-17a7-4a90-91c8-af81b32cce0a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e6971918-17a7-4a90-91c8-af81b32cce0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:32.420245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:32.421206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:32.422033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:32.422033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:32.422033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:32.424816 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 235/462] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:44:32.430745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4de1b-fb9a-4a85-9f81-2053b7397f59 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:32.479885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:32.505495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6971918-17a7-4a90-91c8-af81b32cce0a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:32.545237 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:32.582749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f4de1b-fb9a-4a85-9f81-2053b7397f59 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:32.617839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd6f08-7c6b-4f96-8ddd-949340ab362d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:32.618685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd6f08-7c6b-4f96-8ddd-949340ab362d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:32.619104 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd6f08-7c6b-4f96-8ddd-949340ab362d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:32.619543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd6f08-7c6b-4f96-8ddd-949340ab362d None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:32.620343 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd6f08-7c6b-4f96-8ddd-949340ab362d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:32.620784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd6f08-7c6b-4f96-8ddd-949340ab362d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:32.622291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd6f08-7c6b-4f96-8ddd-949340ab362d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:32.635486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 228/463] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:32.637094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf781af-96a1-4fb1-81c9-224e0d6964b6 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:32.637519 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf781af-96a1-4fb1-81c9-224e0d6964b6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:32.637932 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf781af-96a1-4fb1-81c9-224e0d6964b6 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:32.639780 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 229/464] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:32.646436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd880a04-3045-4356-9b27-689102efbcf4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:32.646726 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd880a04-3045-4356-9b27-689102efbcf4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:32.647935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd880a04-3045-4356-9b27-689102efbcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:32.653844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6971918-17a7-4a90-91c8-af81b32cce0a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:32.668279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-bd880a04-3045-4356-9b27-689102efbcf4 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:32.669621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bd880a04-3045-4356-9b27-689102efbcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:32.697707 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa53eb-76d0-4e43-9782-ef99877ae7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:32.699018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa53eb-76d0-4e43-9782-ef99877ae7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:32.699460 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa53eb-76d0-4e43-9782-ef99877ae7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:32.699903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eaa53eb-76d0-4e43-9782-ef99877ae7b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:32.713772 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa53eb-76d0-4e43-9782-ef99877ae7b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:32.715475 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eaa53eb-76d0-4e43-9782-ef99877ae7b4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4eaa53eb-76d0-4e43-9782-ef99877ae7b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:32.732524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:32.733413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:32.734252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:32.734252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:32.734252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:32.736336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 236/465] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:44:32.736336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa53eb-76d0-4e43-9782-ef99877ae7b4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:32.753661 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 230/466] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:32.754588 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd880a04-3045-4356-9b27-689102efbcf4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:32.780141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:32.822253 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:32.862637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eaa53eb-76d0-4e43-9782-ef99877ae7b4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:32.863926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd880a04-3045-4356-9b27-689102efbcf4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:32.903154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff539b-2b64-4560-87cc-4411e138c784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:32.904178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff539b-2b64-4560-87cc-4411e138c784 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:32.904645 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff539b-2b64-4560-87cc-4411e138c784 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:32.905149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ff539b-2b64-4560-87cc-4411e138c784 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:32.906304 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d3e3b-a173-49c5-8c6e-8605058fa8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:32.907038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d3e3b-a173-49c5-8c6e-8605058fa8b3 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:32.907686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d3e3b-a173-49c5-8c6e-8605058fa8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:32.908069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291d3e3b-a173-49c5-8c6e-8605058fa8b3 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:32.909834 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d3e3b-a173-49c5-8c6e-8605058fa8b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:32.911468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291d3e3b-a173-49c5-8c6e-8605058fa8b3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:32.917748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff539b-2b64-4560-87cc-4411e138c784 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:32.919428 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ff539b-2b64-4560-87cc-4411e138c784 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-83ff539b-2b64-4560-87cc-4411e138c784 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:32.936042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:32.943239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:32.944112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:32.944112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:32.944112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:32.944112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 231/467] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:32.944112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff539b-2b64-4560-87cc-4411e138c784 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:33.023755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 237/468] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:44:32 2022] POST /identity/v3/users => generated 246 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:44:33.025186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d3e3b-a173-49c5-8c6e-8605058fa8b3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:33.040112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:33.091306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ff539b-2b64-4560-87cc-4411e138c784 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:33.134197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb8c18-02ef-4039-b9ae-bc7a1a9287ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:33.134769 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb8c18-02ef-4039-b9ae-bc7a1a9287ef None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.135050 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb8c18-02ef-4039-b9ae-bc7a1a9287ef None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.135320 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb8c18-02ef-4039-b9ae-bc7a1a9287ef None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.136021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb8c18-02ef-4039-b9ae-bc7a1a9287ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:33.136309 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb8c18-02ef-4039-b9ae-bc7a1a9287ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:33.137716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afb8c18-02ef-4039-b9ae-bc7a1a9287ef None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:33.152867 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 232/469] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:33.158320 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb8c18-02ef-4039-b9ae-bc7a1a9287ef None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:33.187098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291d3e3b-a173-49c5-8c6e-8605058fa8b3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:33.219837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:33.232129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba241fa-431c-4299-b2ee-785608ccbd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:33.232836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba241fa-431c-4299-b2ee-785608ccbd6f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.233168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba241fa-431c-4299-b2ee-785608ccbd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.233443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba241fa-431c-4299-b2ee-785608ccbd6f None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.234092 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba241fa-431c-4299-b2ee-785608ccbd6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:33.234398 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba241fa-431c-4299-b2ee-785608ccbd6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:33.235858 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba241fa-431c-4299-b2ee-785608ccbd6f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:33.252163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 238/470] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:33.254270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba241fa-431c-4299-b2ee-785608ccbd6f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:33.282139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb8c18-02ef-4039-b9ae-bc7a1a9287ef None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:33.323938 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc82cdf-1341-4b63-8b22-0091892d9f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:33.324576 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc82cdf-1341-4b63-8b22-0091892d9f13 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.324859 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc82cdf-1341-4b63-8b22-0091892d9f13 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.325134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc82cdf-1341-4b63-8b22-0091892d9f13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.328987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:33.339222 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc82cdf-1341-4b63-8b22-0091892d9f13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:33.343997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc82cdf-1341-4b63-8b22-0091892d9f13 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1bc82cdf-1341-4b63-8b22-0091892d9f13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:33.356640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:33.357561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:33.358372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:33.358372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:33.358372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:33.358372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 233/471] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:33.359026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc82cdf-1341-4b63-8b22-0091892d9f13 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:33.394584 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba241fa-431c-4299-b2ee-785608ccbd6f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:33.436245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf3e07-04e3-4515-8bac-7f9010732a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:33.437051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf3e07-04e3-4515-8bac-7f9010732a70 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.437474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf3e07-04e3-4515-8bac-7f9010732a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.437850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf3e07-04e3-4515-8bac-7f9010732a70 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.438654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf3e07-04e3-4515-8bac-7f9010732a70 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:33.440286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbf3e07-04e3-4515-8bac-7f9010732a70 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:33.481307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc82cdf-1341-4b63-8b22-0091892d9f13 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:33.518695 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994bae6d-80ac-48f0-a80f-50e79700a42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:33.519634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bae6d-80ac-48f0-a80f-50e79700a42a None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.520108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bae6d-80ac-48f0-a80f-50e79700a42a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.520525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994bae6d-80ac-48f0-a80f-50e79700a42a None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.521362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bae6d-80ac-48f0-a80f-50e79700a42a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:33.523039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994bae6d-80ac-48f0-a80f-50e79700a42a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:33.523302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 239/472] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:44:33 2022] POST /identity/v3/users => generated 248 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:44:33.524474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf3e07-04e3-4515-8bac-7f9010732a70 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:33.597043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:33.599980 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 234/473] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:44:33 2022] POST /identity/v3/users => generated 248 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:44:33.601582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b554ac7-5902-4b81-8465-aeb9696912ec None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.601872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b554ac7-5902-4b81-8465-aeb9696912ec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.602151 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b554ac7-5902-4b81-8465-aeb9696912ec None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.607802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 235/474] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:33.610245 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448aab0-8595-4b5c-acaa-6176a6c7636e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.610790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448aab0-8595-4b5c-acaa-6176a6c7636e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.611261 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448aab0-8595-4b5c-acaa-6176a6c7636e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.635637 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8448aab0-8595-4b5c-acaa-6176a6c7636e None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:33.637350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8448aab0-8595-4b5c-acaa-6176a6c7636e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:33.685779 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:33.704445 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf3e07-04e3-4515-8bac-7f9010732a70 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:33.749916 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 236/475] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:33.773397 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a38c06-5385-48ab-9709-269a767a5ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:33.774409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a38c06-5385-48ab-9709-269a767a5ef8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.774896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a38c06-5385-48ab-9709-269a767a5ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.775361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a38c06-5385-48ab-9709-269a767a5ef8 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.776424 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a38c06-5385-48ab-9709-269a767a5ef8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:33.777015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a38c06-5385-48ab-9709-269a767a5ef8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:33.778711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a38c06-5385-48ab-9709-269a767a5ef8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:33.793735 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 240/476] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:33.800296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448aab0-8595-4b5c-acaa-6176a6c7636e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:33.910164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448aab0-8595-4b5c-acaa-6176a6c7636e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:33.946732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1e096-94ac-4a37-a534-d0d71224613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:33.947377 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1e096-94ac-4a37-a534-d0d71224613e None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:33.947609 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1e096-94ac-4a37-a534-d0d71224613e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:33.947912 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e1e096-94ac-4a37-a534-d0d71224613e None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:33.948562 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1e096-94ac-4a37-a534-d0d71224613e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:33.950002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e1e096-94ac-4a37-a534-d0d71224613e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:34.051924 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 237/477] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:44:33 2022] POST /identity/v3/users => generated 249 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:44:34.075129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a38c06-5385-48ab-9709-269a767a5ef8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:34.211545 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a38c06-5385-48ab-9709-269a767a5ef8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:34.214482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:34.255863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820d43c-054f-4c97-9877-c8dae3dba499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:34.256885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820d43c-054f-4c97-9877-c8dae3dba499 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:34.257176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820d43c-054f-4c97-9877-c8dae3dba499 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:34.257461 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820d43c-054f-4c97-9877-c8dae3dba499 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:34.258153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820d43c-054f-4c97-9877-c8dae3dba499 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:34.259542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b820d43c-054f-4c97-9877-c8dae3dba499 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:34.308349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.sql.core [None req-b820d43c-054f-4c97-9877-c8dae3dba499 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 26 17:44:34.308349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 26 17:44:34.308349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [parameters: {'id': '8e2c62a2a2bc4701a6644fcceddbcaad', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 26 17:44:34.308349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81499) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 26 17:44:34.326016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b820d43c-054f-4c97-9877-c8dae3dba499 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 26 17:44:34.326016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:34.326016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 17:44:34.326016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 17:44:34.326016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 17:44:34.326016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 17:44:34.326016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 17:44:34.326016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 17:44:34.326016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 17:44:34.326016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application conn.query(q) Sep 26 17:44:34.326016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 17:44:34.326016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 17:44:34.326016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 17:44:34.326016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application result.read() Sep 26 17:44:34.326016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return ref.to_dict() Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 17:44:34.326890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application next(self.gen) Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application yield resource Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application next(self.gen) Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self.session.rollback() Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self.force_reraise() Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise self.value Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application session.commit() Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 26 17:44:34.328270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._prepare_impl() Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self.session.flush() Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._flush(objects) Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application compat.raise_( Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application flush_context.execute() Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rec.execute(self) Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 26 17:44:34.329349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 17:44:34.330188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application conn.query(q) Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application result.read() Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 17:44:34.331003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [parameters: {'id': '8e2c62a2a2bc4701a6644fcceddbcaad', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:34.331924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 26 17:44:34.333462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:34.334413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 241/478] 10.222.0.67 () {60 vars in 1131 bytes} [Mon Sep 26 17:44:34 2022] POST /identity/v3/roles => generated 144 bytes in 253 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:34.334487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1e096-94ac-4a37-a534-d0d71224613e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:34.373556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:34.483089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e1e096-94ac-4a37-a534-d0d71224613e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:34.528876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a634b76-0ef0-40e0-a69a-0c374a9a4c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:34.529518 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a634b76-0ef0-40e0-a69a-0c374a9a4c43 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:34.529798 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a634b76-0ef0-40e0-a69a-0c374a9a4c43 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:34.530075 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a634b76-0ef0-40e0-a69a-0c374a9a4c43 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:34.545816 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a634b76-0ef0-40e0-a69a-0c374a9a4c43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8a634b76-0ef0-40e0-a69a-0c374a9a4c43 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:34.551824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:34.554087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:34.556008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:34.556008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:34.556008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:34.556008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:34.556008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:34.556008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:34.556008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:34.556008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:34.556008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:34.556008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:34.556008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 26 17:44:34.556008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:34.556008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 238/479] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:44:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:34.568887 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820d43c-054f-4c97-9877-c8dae3dba499 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:34.719159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820d43c-054f-4c97-9877-c8dae3dba499 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:34.776041 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9d695-393a-431d-84af-1d675ccbb318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:34.776572 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9d695-393a-431d-84af-1d675ccbb318 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:34.776852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9d695-393a-431d-84af-1d675ccbb318 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:34.777122 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9d695-393a-431d-84af-1d675ccbb318 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:34.777828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9d695-393a-431d-84af-1d675ccbb318 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:34.778134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9d695-393a-431d-84af-1d675ccbb318 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:34.779494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb9d695-393a-431d-84af-1d675ccbb318 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:34.794948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 242/480] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:35.519637 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c633c-eeeb-45cc-b84a-10f8bf5c9aab None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:35.520156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c633c-eeeb-45cc-b84a-10f8bf5c9aab None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:35.520259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c633c-eeeb-45cc-b84a-10f8bf5c9aab None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:35.521823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 239/481] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:35.527082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298c172-7f0e-4751-87ec-9c25978cd48b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:35.527368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298c172-7f0e-4751-87ec-9c25978cd48b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:35.527657 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8298c172-7f0e-4751-87ec-9c25978cd48b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:35.550851 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8298c172-7f0e-4751-87ec-9c25978cd48b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:35.556881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8298c172-7f0e-4751-87ec-9c25978cd48b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:35.591252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:35.652862 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 243/482] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:36.020355 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c633c-eeeb-45cc-b84a-10f8bf5c9aab None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:36.152052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:36.173453 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c633c-eeeb-45cc-b84a-10f8bf5c9aab None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:36.222131 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de13ed-bcbe-439a-81f5-701994370399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:36.222765 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de13ed-bcbe-439a-81f5-701994370399 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:36.223040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de13ed-bcbe-439a-81f5-701994370399 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:36.223302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de13ed-bcbe-439a-81f5-701994370399 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:36.239364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de13ed-bcbe-439a-81f5-701994370399 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-00de13ed-bcbe-439a-81f5-701994370399 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:36.244112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:36.244967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:36.245741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:36.245741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:36.245741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:36.245741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:36.245741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:36.245741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:36.245741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:36.245741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:36.245741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:36.245741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:36.245741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:36.245741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:36.245741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 240/483] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:36.251127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298c172-7f0e-4751-87ec-9c25978cd48b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:36.337936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be604d07-b232-4adc-88f4-30e5c06ccd39 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:36.338757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be604d07-b232-4adc-88f4-30e5c06ccd39 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:36.339034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be604d07-b232-4adc-88f4-30e5c06ccd39 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:36.340961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 241/484] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:36.347171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6781330-784f-4985-aef8-80b44eff10a5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:36.347452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6781330-784f-4985-aef8-80b44eff10a5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:36.347767 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6781330-784f-4985-aef8-80b44eff10a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:36.370593 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a6781330-784f-4985-aef8-80b44eff10a5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:36.372062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a6781330-784f-4985-aef8-80b44eff10a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:36.437256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8298c172-7f0e-4751-87ec-9c25978cd48b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:36.479468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10578e1-69d1-48a9-b322-7ea9c1792e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:36.480580 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10578e1-69d1-48a9-b322-7ea9c1792e94 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:36.481061 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10578e1-69d1-48a9-b322-7ea9c1792e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:36.481548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10578e1-69d1-48a9-b322-7ea9c1792e94 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:36.482421 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10578e1-69d1-48a9-b322-7ea9c1792e94 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:36.482919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10578e1-69d1-48a9-b322-7ea9c1792e94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:36.484710 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10578e1-69d1-48a9-b322-7ea9c1792e94 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:36.488211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 242/485] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:36.494677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 244/486] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:44:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:36.500456 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6781330-784f-4985-aef8-80b44eff10a5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:36.625592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:36.649452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6781330-784f-4985-aef8-80b44eff10a5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:36.653808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda926d-3cdd-496b-9764-13f1983e3ad9 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:36.654331 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda926d-3cdd-496b-9764-13f1983e3ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:36.654805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda926d-3cdd-496b-9764-13f1983e3ad9 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:36.656718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 245/487] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:36.663619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5e0e7-9827-4058-920d-e0803518e3d4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:36.664452 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5e0e7-9827-4058-920d-e0803518e3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:36.665404 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef5e0e7-9827-4058-920d-e0803518e3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:36.687180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8470d03-e399-4674-a7b6-d8047047a5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:36.687901 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8470d03-e399-4674-a7b6-d8047047a5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:36.688176 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8470d03-e399-4674-a7b6-d8047047a5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:36.688444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8470d03-e399-4674-a7b6-d8047047a5a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:36.689265 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fef5e0e7-9827-4058-920d-e0803518e3d4 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:36.691671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fef5e0e7-9827-4058-920d-e0803518e3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:36.700252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8470d03-e399-4674-a7b6-d8047047a5a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:36.701588 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8470d03-e399-4674-a7b6-d8047047a5a2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:36.717890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e8470d03-e399-4674-a7b6-d8047047a5a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:36.727427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:36.736770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:36.738069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:36.738069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:36.738069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:36.738069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 243/488] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:36.738069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8470d03-e399-4674-a7b6-d8047047a5a2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:36.792024 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 246/489] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:36.876163 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8470d03-e399-4674-a7b6-d8047047a5a2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:36.887894 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9b15-8796-441f-b6b0-6c0f364448e5 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:36.888223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9b15-8796-441f-b6b0-6c0f364448e5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:36.888440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9b15-8796-441f-b6b0-6c0f364448e5 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:36.890672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 247/490] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:44:36.896235 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3ad9d-72a5-4864-a7a0-01fea61e2ca2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:36.896792 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3ad9d-72a5-4864-a7a0-01fea61e2ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:36.897551 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f3ad9d-72a5-4864-a7a0-01fea61e2ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:36.911144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:36.917181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959df8f-a487-45e8-a58f-47fca00353c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:36.917822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959df8f-a487-45e8-a58f-47fca00353c2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:36.918204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959df8f-a487-45e8-a58f-47fca00353c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:36.918379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959df8f-a487-45e8-a58f-47fca00353c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:36.919002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959df8f-a487-45e8-a58f-47fca00353c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:36.919299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959df8f-a487-45e8-a58f-47fca00353c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:36.920774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959df8f-a487-45e8-a58f-47fca00353c2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:36.924719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-87f3ad9d-72a5-4864-a7a0-01fea61e2ca2 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:36.926415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-87f3ad9d-72a5-4864-a7a0-01fea61e2ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:36.934735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 244/491] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:36.940458 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959df8f-a487-45e8-a58f-47fca00353c2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:37.021947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 248/492] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:37.023082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3ad9d-72a5-4864-a7a0-01fea61e2ca2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:37.076613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959df8f-a487-45e8-a58f-47fca00353c2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:37.119323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:37.132579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea360b59-f7d6-4651-bd4d-4998dd95e248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:37.137767 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea360b59-f7d6-4651-bd4d-4998dd95e248 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:37.137767 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea360b59-f7d6-4651-bd4d-4998dd95e248 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:37.137767 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea360b59-f7d6-4651-bd4d-4998dd95e248 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:37.141296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f3ad9d-72a5-4864-a7a0-01fea61e2ca2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:37.141676 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea360b59-f7d6-4651-bd4d-4998dd95e248 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:37.143917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea360b59-f7d6-4651-bd4d-4998dd95e248 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ea360b59-f7d6-4651-bd4d-4998dd95e248 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:37.157184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:37.158295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:37.160337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:37.160337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:37.160337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:37.160337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:37.160337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:37.160337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:37.160337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:37.160337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:37.160337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:44:37.160337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:37.162515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 245/493] 10.222.0.67 () {56 vars in 1093 bytes} [Mon Sep 26 17:44:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:44:37.167714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea360b59-f7d6-4651-bd4d-4998dd95e248 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:37.185159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b1234-b39c-42bf-91ac-73426ec56c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:37.186122 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b1234-b39c-42bf-91ac-73426ec56c99 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:37.186411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b1234-b39c-42bf-91ac-73426ec56c99 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:37.186690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9b1234-b39c-42bf-91ac-73426ec56c99 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:37.202747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9b1234-b39c-42bf-91ac-73426ec56c99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6f9b1234-b39c-42bf-91ac-73426ec56c99 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:37.209813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:37.210692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:37.211560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:37.211560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:37.211560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:37.211560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:37.211560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:37.211560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:37.211560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:37.211560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:37.211560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:37.211560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:37.211560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:37.211560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:37.211560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 249/494] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:37.212213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b1234-b39c-42bf-91ac-73426ec56c99 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:37.227691 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:37.282066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea360b59-f7d6-4651-bd4d-4998dd95e248 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:37.318444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6590766-aa11-4cbc-82c4-ad7c0cad658d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:37.319117 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6590766-aa11-4cbc-82c4-ad7c0cad658d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:37.319402 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6590766-aa11-4cbc-82c4-ad7c0cad658d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:37.319704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6590766-aa11-4cbc-82c4-ad7c0cad658d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:37.325162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9b1234-b39c-42bf-91ac-73426ec56c99 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:37.330426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6590766-aa11-4cbc-82c4-ad7c0cad658d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b6590766-aa11-4cbc-82c4-ad7c0cad658d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:37.336905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:37.337771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:37.338547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:37.338547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:37.338547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:37.338547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:37.338547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:37.338547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:37.338547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:37.338547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:37.338547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:37.338547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:37.338547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:37.338547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:37.338547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 246/495] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:37.339192 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6590766-aa11-4cbc-82c4-ad7c0cad658d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:37.368310 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b038827-9e03-41de-824c-1c4c9916696d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:37.368985 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b038827-9e03-41de-824c-1c4c9916696d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:37.369266 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b038827-9e03-41de-824c-1c4c9916696d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:37.369544 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b038827-9e03-41de-824c-1c4c9916696d None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:37.370429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b038827-9e03-41de-824c-1c4c9916696d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:37.370809 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b038827-9e03-41de-824c-1c4c9916696d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:37.372453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b038827-9e03-41de-824c-1c4c9916696d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:37.410702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:37.415191 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 250/496] 10.222.0.67 () {56 vars in 1133 bytes} [Mon Sep 26 17:44:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:37.416030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b038827-9e03-41de-824c-1c4c9916696d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:37.448412 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6590766-aa11-4cbc-82c4-ad7c0cad658d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:37.460632 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:37.489012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa26188-1ff8-4843-bf00-900487387683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:37.489623 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa26188-1ff8-4843-bf00-900487387683 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:37.489904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa26188-1ff8-4843-bf00-900487387683 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:37.490179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa26188-1ff8-4843-bf00-900487387683 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:37.490848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa26188-1ff8-4843-bf00-900487387683 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:37.491154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa26188-1ff8-4843-bf00-900487387683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:37.492569 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa26188-1ff8-4843-bf00-900487387683 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:37.505809 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 247/497] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:44:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:37.505809 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa26188-1ff8-4843-bf00-900487387683 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:37.532137 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b038827-9e03-41de-824c-1c4c9916696d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:37.569852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8371e-bf00-4497-9329-9866862b1344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:37.570511 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8371e-bf00-4497-9329-9866862b1344 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:37.570846 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8371e-bf00-4497-9329-9866862b1344 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:37.571130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c8371e-bf00-4497-9329-9866862b1344 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:37.571821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8371e-bf00-4497-9329-9866862b1344 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:37.572171 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8371e-bf00-4497-9329-9866862b1344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:37.573795 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c8371e-bf00-4497-9329-9866862b1344 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:37.587823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 251/498] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:44:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:37.587823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8371e-bf00-4497-9329-9866862b1344 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:37.614931 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa26188-1ff8-4843-bf00-900487387683 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:37.649564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1601f42-e0f3-427e-a42f-586fa0831b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:37.650193 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1601f42-e0f3-427e-a42f-586fa0831b17 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:37.650457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1601f42-e0f3-427e-a42f-586fa0831b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:37.650726 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1601f42-e0f3-427e-a42f-586fa0831b17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:37.658035 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:37.661829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1601f42-e0f3-427e-a42f-586fa0831b17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b1601f42-e0f3-427e-a42f-586fa0831b17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:37.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:37.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:37.675258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:37.675258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:37.675258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:37.675258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:37.675258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:37.675258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:37.675258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:37.675258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:37.675258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:37.675258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:37.675258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:37.675258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:37.675258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 248/499] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:37.675258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1601f42-e0f3-427e-a42f-586fa0831b17 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:37.698499 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c8371e-bf00-4497-9329-9866862b1344 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:37.712153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:37.739025 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e00022-0821-477f-ad75-d4c312b5f97d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:37.739666 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e00022-0821-477f-ad75-d4c312b5f97d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:37.740037 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e00022-0821-477f-ad75-d4c312b5f97d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:37.740310 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e00022-0821-477f-ad75-d4c312b5f97d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:37.786559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1601f42-e0f3-427e-a42f-586fa0831b17 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:37.812947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e00022-0821-477f-ad75-d4c312b5f97d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:37.814412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e00022-0821-477f-ad75-d4c312b5f97d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:37.821758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7bc5a-275f-4163-ad69-ce472e2e3b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:37.822447 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7bc5a-275f-4163-ad69-ce472e2e3b5a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:37.822724 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7bc5a-275f-4163-ad69-ce472e2e3b5a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:37.822992 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7bc5a-275f-4163-ad69-ce472e2e3b5a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:37.833711 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7bc5a-275f-4163-ad69-ce472e2e3b5a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:37.835179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7bc5a-275f-4163-ad69-ce472e2e3b5a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c1a7bc5a-275f-4163-ad69-ce472e2e3b5a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:37.850204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:37.851260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:37.852474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:37.852474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:37.852474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:37.852474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 249/500] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:37.852474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7bc5a-275f-4163-ad69-ce472e2e3b5a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:37.878922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:37.918978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 252/501] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:37.919837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e00022-0821-477f-ad75-d4c312b5f97d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:37.941251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:37.956923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7bc5a-275f-4163-ad69-ce472e2e3b5a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:37.993514 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe338c0d-e6ea-4865-98bf-1fa2533be4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:37.994154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe338c0d-e6ea-4865-98bf-1fa2533be4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:37.994437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe338c0d-e6ea-4865-98bf-1fa2533be4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:37.994713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe338c0d-e6ea-4865-98bf-1fa2533be4c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.007350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe338c0d-e6ea-4865-98bf-1fa2533be4c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.008847 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe338c0d-e6ea-4865-98bf-1fa2533be4c8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fe338c0d-e6ea-4865-98bf-1fa2533be4c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:38.023334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:38.024445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:38.030250 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:38.030250 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:38.030250 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:38.030449 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e00022-0821-477f-ad75-d4c312b5f97d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.030532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 250/502] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:38.030532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e147f3b-1335-4ebb-992d-cc39cbb30854 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.030532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e147f3b-1335-4ebb-992d-cc39cbb30854 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.030532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e147f3b-1335-4ebb-992d-cc39cbb30854 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.030532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 251/503] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:38.030532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e147f3b-1335-4ebb-992d-cc39cbb30854 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.059374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23f74b-f07f-4813-b606-1ab47c028675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.060012 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23f74b-f07f-4813-b606-1ab47c028675 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.060302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23f74b-f07f-4813-b606-1ab47c028675 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.060572 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23f74b-f07f-4813-b606-1ab47c028675 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.061245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23f74b-f07f-4813-b606-1ab47c028675 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:38.061538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23f74b-f07f-4813-b606-1ab47c028675 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.062896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23f74b-f07f-4813-b606-1ab47c028675 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.071281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 253/504] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:44:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:38.072098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23f74b-f07f-4813-b606-1ab47c028675 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.096953 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:38.135285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e147f3b-1335-4ebb-992d-cc39cbb30854 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.169454 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58315589-7392-4b19-a220-2d40a226307b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.169833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23f74b-f07f-4813-b606-1ab47c028675 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.170049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58315589-7392-4b19-a220-2d40a226307b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.170331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58315589-7392-4b19-a220-2d40a226307b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.170598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58315589-7392-4b19-a220-2d40a226307b None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.171228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58315589-7392-4b19-a220-2d40a226307b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:38.171525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58315589-7392-4b19-a220-2d40a226307b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.172916 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58315589-7392-4b19-a220-2d40a226307b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.176123 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:38.185930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 252/505] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:38.190373 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58315589-7392-4b19-a220-2d40a226307b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.203369 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357aead3-0f9c-4f4e-b77a-9bb3d22db823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.204022 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357aead3-0f9c-4f4e-b77a-9bb3d22db823 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.204301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357aead3-0f9c-4f4e-b77a-9bb3d22db823 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.204572 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357aead3-0f9c-4f4e-b77a-9bb3d22db823 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.215097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357aead3-0f9c-4f4e-b77a-9bb3d22db823 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.216578 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357aead3-0f9c-4f4e-b77a-9bb3d22db823 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-357aead3-0f9c-4f4e-b77a-9bb3d22db823 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:38.231439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:38.232428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:38.233256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:38.233256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:38.233256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:38.233256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 254/506] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:38.233906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f5e61-02e1-41f9-b92a-6dafb61d1fcb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.234016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f5e61-02e1-41f9-b92a-6dafb61d1fcb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.234205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f5e61-02e1-41f9-b92a-6dafb61d1fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.252463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fb2f5e61-02e1-41f9-b92a-6dafb61d1fcb None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:38.253817 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fb2f5e61-02e1-41f9-b92a-6dafb61d1fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:38.294412 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58315589-7392-4b19-a220-2d40a226307b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.319596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 255/507] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:38.320450 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f5e61-02e1-41f9-b92a-6dafb61d1fcb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.329464 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204179c0-3ce6-4c48-ab19-8e23e72b6e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.330069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204179c0-3ce6-4c48-ab19-8e23e72b6e18 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.330498 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204179c0-3ce6-4c48-ab19-8e23e72b6e18 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.330903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204179c0-3ce6-4c48-ab19-8e23e72b6e18 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.331599 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204179c0-3ce6-4c48-ab19-8e23e72b6e18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:38.331934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204179c0-3ce6-4c48-ab19-8e23e72b6e18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.333298 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204179c0-3ce6-4c48-ab19-8e23e72b6e18 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.334261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:38.344653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 253/508] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:38.345576 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204179c0-3ce6-4c48-ab19-8e23e72b6e18 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.405070 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:38.445086 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f5e61-02e1-41f9-b92a-6dafb61d1fcb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.453006 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204179c0-3ce6-4c48-ab19-8e23e72b6e18 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.482510 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54d4eb-cf9c-435e-86e4-8a9d040fb739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.483406 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54d4eb-cf9c-435e-86e4-8a9d040fb739 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.483743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54d4eb-cf9c-435e-86e4-8a9d040fb739 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.483979 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b54d4eb-cf9c-435e-86e4-8a9d040fb739 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.489943 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c6335-da29-49e9-b31f-4b8417dd3d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.490568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c6335-da29-49e9-b31f-4b8417dd3d09 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.490842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c6335-da29-49e9-b31f-4b8417dd3d09 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.491109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43c6335-da29-49e9-b31f-4b8417dd3d09 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.495174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54d4eb-cf9c-435e-86e4-8a9d040fb739 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.496662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b54d4eb-cf9c-435e-86e4-8a9d040fb739 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.498645 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c6335-da29-49e9-b31f-4b8417dd3d09 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.500087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43c6335-da29-49e9-b31f-4b8417dd3d09 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d43c6335-da29-49e9-b31f-4b8417dd3d09 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:38.512059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:38.512945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:38.523313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:38.523313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:38.523313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:38.523313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:38.523313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:38.523313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:38.523313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:38.523313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:38.523313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:44:38.523313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-7b54d4eb-cf9c-435e-86e4-8a9d040fb739 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:38.523978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:38.524897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:38.525957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:38.525957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:38.525957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:38.526147 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 254/509] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:38.526231 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 256/510] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:38.526308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c6335-da29-49e9-b31f-4b8417dd3d09 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.526384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54d4eb-cf9c-435e-86e4-8a9d040fb739 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.587938 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:38.634288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b54d4eb-cf9c-435e-86e4-8a9d040fb739 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.637514 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43c6335-da29-49e9-b31f-4b8417dd3d09 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.653948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:38.668404 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d0656-aa83-4875-9f6d-0e7d169183fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.668975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d0656-aa83-4875-9f6d-0e7d169183fc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.669259 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d0656-aa83-4875-9f6d-0e7d169183fc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.669532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d0656-aa83-4875-9f6d-0e7d169183fc None admin] PATH_INFO: `/v3/users/glance` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.672051 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187b572-ee2e-4466-ae32-5e7607667872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.672656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187b572-ee2e-4466-ae32-5e7607667872 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.673102 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187b572-ee2e-4466-ae32-5e7607667872 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.673740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187b572-ee2e-4466-ae32-5e7607667872 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.675095 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187b572-ee2e-4466-ae32-5e7607667872 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:38.675735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187b572-ee2e-4466-ae32-5e7607667872 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.678110 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d0656-aa83-4875-9f6d-0e7d169183fc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.678472 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187b572-ee2e-4466-ae32-5e7607667872 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.679546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d0656-aa83-4875-9f6d-0e7d169183fc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.690743 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 255/511] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:38.691542 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187b572-ee2e-4466-ae32-5e7607667872 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-851d0656-aa83-4875-9f6d-0e7d169183fc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:38.693203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:38.694073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:38.694920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:38.694920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:38.694920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:38.694920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:38.694920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:38.694920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:38.694920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:38.694920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:38.694920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 26 17:44:38.694920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:38.694920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 257/512] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:38.695447 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d0656-aa83-4875-9f6d-0e7d169183fc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.793610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187b572-ee2e-4466-ae32-5e7607667872 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.800271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d0656-aa83-4875-9f6d-0e7d169183fc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.817440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:38.827458 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a4e07-54ff-4a27-80e3-2e0f7ddf0759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.828079 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a4e07-54ff-4a27-80e3-2e0f7ddf0759 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.833387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a4e07-54ff-4a27-80e3-2e0f7ddf0759 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.833387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a4e07-54ff-4a27-80e3-2e0f7ddf0759 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.833387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a4e07-54ff-4a27-80e3-2e0f7ddf0759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:38.833387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a4e07-54ff-4a27-80e3-2e0f7ddf0759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.833387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a4e07-54ff-4a27-80e3-2e0f7ddf0759 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.834920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157536c6-7a14-490d-b14e-ba6586c7869c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.835516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157536c6-7a14-490d-b14e-ba6586c7869c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.835828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157536c6-7a14-490d-b14e-ba6586c7869c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.836097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157536c6-7a14-490d-b14e-ba6586c7869c None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.836800 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157536c6-7a14-490d-b14e-ba6586c7869c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:38.837100 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157536c6-7a14-490d-b14e-ba6586c7869c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.838576 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157536c6-7a14-490d-b14e-ba6586c7869c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.842699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 256/513] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:38.843485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a4e07-54ff-4a27-80e3-2e0f7ddf0759 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.857885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 258/514] 10.222.0.67 () {56 vars in 1137 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:38.858697 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157536c6-7a14-490d-b14e-ba6586c7869c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:38.880458 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:38.946035 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a4e07-54ff-4a27-80e3-2e0f7ddf0759 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.961419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157536c6-7a14-490d-b14e-ba6586c7869c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:38.984908 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ecb0ae-8e48-42d7-8fb5-30459e0f47cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:38.985539 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ecb0ae-8e48-42d7-8fb5-30459e0f47cd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:38.985817 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ecb0ae-8e48-42d7-8fb5-30459e0f47cd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:38.986089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ecb0ae-8e48-42d7-8fb5-30459e0f47cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:38.997749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ecb0ae-8e48-42d7-8fb5-30459e0f47cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:38.999182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ecb0ae-8e48-42d7-8fb5-30459e0f47cd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:38.999521 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961d127-0d18-4e08-b567-02c891b23ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:39.000149 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961d127-0d18-4e08-b567-02c891b23ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:39.000427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961d127-0d18-4e08-b567-02c891b23ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:39.000697 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f961d127-0d18-4e08-b567-02c891b23ce7 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:39.001410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961d127-0d18-4e08-b567-02c891b23ce7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:39.001711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961d127-0d18-4e08-b567-02c891b23ce7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:39.003477 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f961d127-0d18-4e08-b567-02c891b23ce7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-10ecb0ae-8e48-42d7-8fb5-30459e0f47cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:39.013566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:39.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:39.015873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:39.015873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:39.015873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:39.015873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 257/515] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:39.015873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ecb0ae-8e48-42d7-8fb5-30459e0f47cd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:39.026474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 259/516] 10.222.0.67 () {56 vars in 1137 bytes} [Mon Sep 26 17:44:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:39.026474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961d127-0d18-4e08-b567-02c891b23ce7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:39.060217 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:39.106589 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:39.118824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ecb0ae-8e48-42d7-8fb5-30459e0f47cd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:39.125335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f961d127-0d18-4e08-b567-02c891b23ce7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:39.151898 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fba29-0586-4daa-bfb1-1d93d1d730d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:39.152510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fba29-0586-4daa-bfb1-1d93d1d730d8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:39.152789 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fba29-0586-4daa-bfb1-1d93d1d730d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:39.153058 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fba29-0586-4daa-bfb1-1d93d1d730d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:39.159663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977448b-2991-408f-97dc-a9a8683d95dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:39.160237 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fba29-0586-4daa-bfb1-1d93d1d730d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:39.160326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977448b-2991-408f-97dc-a9a8683d95dc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:39.160570 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977448b-2991-408f-97dc-a9a8683d95dc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:39.160844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977448b-2991-408f-97dc-a9a8683d95dc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:39.161681 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fba29-0586-4daa-bfb1-1d93d1d730d8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:39.167715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977448b-2991-408f-97dc-a9a8683d95dc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:39.169125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f977448b-2991-408f-97dc-a9a8683d95dc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:39.173221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-147fba29-0586-4daa-bfb1-1d93d1d730d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:39.173221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:39.173221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:39.173221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:39.173221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:39.173221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:39.173221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:39.173221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:39.173221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:39.173221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:39.173221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:39.173221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:39.173221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:39.173221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:39.173221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:39.174556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:39.175411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:39.175411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:39.175411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:39.175411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:39.175411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:39.175411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:39.175411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:39.175411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:39.175411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 258/517] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:39.175898 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fba29-0586-4daa-bfb1-1d93d1d730d8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f977448b-2991-408f-97dc-a9a8683d95dc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:39.185154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:39.186214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:39.187073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:39.187073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:39.187073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:39.187073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:39.187073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:39.187073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:39.187073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:39.187073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:39.187073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 17:44:39.187073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:39.187073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 260/518] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:39.187073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977448b-2991-408f-97dc-a9a8683d95dc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:39.279094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fba29-0586-4daa-bfb1-1d93d1d730d8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:39.280639 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:39.288393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977448b-2991-408f-97dc-a9a8683d95dc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:39.314015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535a4e7-9684-469d-9218-2690ef2b31be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:39.315352 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535a4e7-9684-469d-9218-2690ef2b31be None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:39.315900 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535a4e7-9684-469d-9218-2690ef2b31be None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:39.316916 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7535a4e7-9684-469d-9218-2690ef2b31be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:39.323466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c80e0f-2cbe-46c1-ab61-2deabdf1ca80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:39.324093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c80e0f-2cbe-46c1-ab61-2deabdf1ca80 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:39.324373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c80e0f-2cbe-46c1-ab61-2deabdf1ca80 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:39.324643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c80e0f-2cbe-46c1-ab61-2deabdf1ca80 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:39.325277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c80e0f-2cbe-46c1-ab61-2deabdf1ca80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:39.325576 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c80e0f-2cbe-46c1-ab61-2deabdf1ca80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:39.326930 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c80e0f-2cbe-46c1-ab61-2deabdf1ca80 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:39.334609 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:39.339140 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 261/519] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:39.339975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c80e0f-2cbe-46c1-ab61-2deabdf1ca80 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:39.386097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535a4e7-9684-469d-9218-2690ef2b31be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:39.387550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7535a4e7-9684-469d-9218-2690ef2b31be None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:39.443595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c80e0f-2cbe-46c1-ab61-2deabdf1ca80 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:39.478425 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19791f4a-bace-4a82-8da2-81ae350d4105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:39.479026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19791f4a-bace-4a82-8da2-81ae350d4105 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:39.479305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19791f4a-bace-4a82-8da2-81ae350d4105 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:39.479578 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19791f4a-bace-4a82-8da2-81ae350d4105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:39.485692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 259/520] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:39.486623 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535a4e7-9684-469d-9218-2690ef2b31be None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:39.503605 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:39.538570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:39.548169 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19791f4a-bace-4a82-8da2-81ae350d4105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:39.550407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19791f4a-bace-4a82-8da2-81ae350d4105 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:39.589348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7535a4e7-9684-469d-9218-2690ef2b31be None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:39.624423 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc4286-cc45-405a-a344-8d5af9bf8561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:39.625018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc4286-cc45-405a-a344-8d5af9bf8561 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:39.625294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc4286-cc45-405a-a344-8d5af9bf8561 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:39.625561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1dc4286-cc45-405a-a344-8d5af9bf8561 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:39.626239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc4286-cc45-405a-a344-8d5af9bf8561 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:39.626535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc4286-cc45-405a-a344-8d5af9bf8561 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:39.628640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1dc4286-cc45-405a-a344-8d5af9bf8561 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:39.646858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 260/521] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:39.647457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc4286-cc45-405a-a344-8d5af9bf8561 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:39.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 262/522] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:39.651308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19791f4a-bace-4a82-8da2-81ae350d4105 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:39.704818 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:39.749781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19791f4a-bace-4a82-8da2-81ae350d4105 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:39.750229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1dc4286-cc45-405a-a344-8d5af9bf8561 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:39.759456 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:39.784077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba21131-0f17-46d0-a34e-ccdc0c0cab64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:39.784769 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509e71f-36ae-4e70-b823-c11d2f2e6b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:39.784914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba21131-0f17-46d0-a34e-ccdc0c0cab64 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:39.785109 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba21131-0f17-46d0-a34e-ccdc0c0cab64 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:39.785387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba21131-0f17-46d0-a34e-ccdc0c0cab64 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:39.785474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509e71f-36ae-4e70-b823-c11d2f2e6b3c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:39.785670 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509e71f-36ae-4e70-b823-c11d2f2e6b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:39.785934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509e71f-36ae-4e70-b823-c11d2f2e6b3c None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:39.786028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba21131-0f17-46d0-a34e-ccdc0c0cab64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:39.786653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509e71f-36ae-4e70-b823-c11d2f2e6b3c None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:39.787014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509e71f-36ae-4e70-b823-c11d2f2e6b3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:39.787486 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba21131-0f17-46d0-a34e-ccdc0c0cab64 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:39.788865 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509e71f-36ae-4e70-b823-c11d2f2e6b3c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:39.824761 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 261/523] 10.222.0.67 () {56 vars in 1139 bytes} [Mon Sep 26 17:44:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:39.825581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509e71f-36ae-4e70-b823-c11d2f2e6b3c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:39.856566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 263/524] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:44:39 2022] POST /identity/v3/users => generated 247 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:44:39.857382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba21131-0f17-46d0-a34e-ccdc0c0cab64 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:39.943775 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509e71f-36ae-4e70-b823-c11d2f2e6b3c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:39.991616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba21131-0f17-46d0-a34e-ccdc0c0cab64 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:39.996467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918029f-7abd-484c-8456-e8eaf86384ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:39.997121 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918029f-7abd-484c-8456-e8eaf86384ef None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:39.997403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918029f-7abd-484c-8456-e8eaf86384ef None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:39.997673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918029f-7abd-484c-8456-e8eaf86384ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:40.004382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:40.011579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918029f-7abd-484c-8456-e8eaf86384ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:40.013222 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918029f-7abd-484c-8456-e8eaf86384ef None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7918029f-7abd-484c-8456-e8eaf86384ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:40.030732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:40.031708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:40.032760 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:40.032760 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:40.032760 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:40.032760 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 262/525] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:40.034015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918029f-7abd-484c-8456-e8eaf86384ef None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:40.037041 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c375e5-d7ef-4ad1-8758-b662a49b768a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:40.038167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c375e5-d7ef-4ad1-8758-b662a49b768a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:40.038815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c375e5-d7ef-4ad1-8758-b662a49b768a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:40.039512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c375e5-d7ef-4ad1-8758-b662a49b768a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:40.040245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c375e5-d7ef-4ad1-8758-b662a49b768a None admin] RBAC: Adding query filter params (role.id=90322db354f94f03b32ec088922baf2e, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=b9ed29a3f6be4d64bb3751f9e95eb56b) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:40.040557 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c375e5-d7ef-4ad1-8758-b662a49b768a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:40.042016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c375e5-d7ef-4ad1-8758-b662a49b768a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:40.047778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 264/526] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:39 2022] GET /identity/v3/role_assignments?user.id=b9ed29a3f6be4d64bb3751f9e95eb56b&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=90322db354f94f03b32ec088922baf2e => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:40.048587 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c375e5-d7ef-4ad1-8758-b662a49b768a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:40.057474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:40.166693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918029f-7abd-484c-8456-e8eaf86384ef None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:40.179731 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c375e5-d7ef-4ad1-8758-b662a49b768a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:40.219905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575908aa-fe5b-4a3a-8438-aa6bd86f2cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:40.220553 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575908aa-fe5b-4a3a-8438-aa6bd86f2cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:40.220917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575908aa-fe5b-4a3a-8438-aa6bd86f2cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:40.221188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575908aa-fe5b-4a3a-8438-aa6bd86f2cb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:40.226672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27fd74-5ac2-4bc1-a016-869fc64da534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:40.227719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27fd74-5ac2-4bc1-a016-869fc64da534 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:40.228204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27fd74-5ac2-4bc1-a016-869fc64da534 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:40.228691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27fd74-5ac2-4bc1-a016-869fc64da534 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:40.233570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575908aa-fe5b-4a3a-8438-aa6bd86f2cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:40.235058 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575908aa-fe5b-4a3a-8438-aa6bd86f2cb7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-575908aa-fe5b-4a3a-8438-aa6bd86f2cb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:40.258348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:40.259297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:40.260229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:40.260229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:40.260229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:40.260229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 263/527] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:40.260870 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575908aa-fe5b-4a3a-8438-aa6bd86f2cb7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:40.295384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:40.326561 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27fd74-5ac2-4bc1-a016-869fc64da534 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:40.328367 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27fd74-5ac2-4bc1-a016-869fc64da534 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:40.365586 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:40.406825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575908aa-fe5b-4a3a-8438-aa6bd86f2cb7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:40.443342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d330bf-0936-45ee-826b-cd6ee8669c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:40.444002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d330bf-0936-45ee-826b-cd6ee8669c9e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:40.444284 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d330bf-0936-45ee-826b-cd6ee8669c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:40.444564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d330bf-0936-45ee-826b-cd6ee8669c9e None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:40.445278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d330bf-0936-45ee-826b-cd6ee8669c9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:40.445504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d330bf-0936-45ee-826b-cd6ee8669c9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:40.446877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d330bf-0936-45ee-826b-cd6ee8669c9e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:40.455159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 265/528] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:40.457457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27fd74-5ac2-4bc1-a016-869fc64da534 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:40.469225 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 264/529] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:40.469225 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d330bf-0936-45ee-826b-cd6ee8669c9e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:40.562721 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:40.579858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d330bf-0936-45ee-826b-cd6ee8669c9e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:40.594082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27fd74-5ac2-4bc1-a016-869fc64da534 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:40.617582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c88507-fbb0-4819-b553-0d1fa2336d32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:40.618216 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c88507-fbb0-4819-b553-0d1fa2336d32 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:40.618495 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c88507-fbb0-4819-b553-0d1fa2336d32 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:40.618768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c88507-fbb0-4819-b553-0d1fa2336d32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:40.628044 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:40.632596 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c88507-fbb0-4819-b553-0d1fa2336d32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:40.634181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c88507-fbb0-4819-b553-0d1fa2336d32 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:40.637785 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376a8bd-37f2-4a18-8228-c1d98be1f450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:40.638624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376a8bd-37f2-4a18-8228-c1d98be1f450 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:40.639481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376a8bd-37f2-4a18-8228-c1d98be1f450 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:40.646175 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d376a8bd-37f2-4a18-8228-c1d98be1f450 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:40.646175 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376a8bd-37f2-4a18-8228-c1d98be1f450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:40.646175 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376a8bd-37f2-4a18-8228-c1d98be1f450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:40.646175 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d376a8bd-37f2-4a18-8228-c1d98be1f450 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f4c88507-fbb0-4819-b553-0d1fa2336d32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:40.650504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:40.651469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:40.652391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:40.652391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:40.652391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:40.656008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 265/530] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:40.657034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c88507-fbb0-4819-b553-0d1fa2336d32 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:40.661273 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 266/531] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:40.662414 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376a8bd-37f2-4a18-8228-c1d98be1f450 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:40.778269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d376a8bd-37f2-4a18-8228-c1d98be1f450 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:40.781556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c88507-fbb0-4819-b553-0d1fa2336d32 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:40.820773 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f88cd-8189-4d7a-8c8f-a43b59bb4289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:40.821296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2740b-27b6-4cd7-bd04-8de6d981c683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:40.821923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2740b-27b6-4cd7-bd04-8de6d981c683 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:40.822225 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f88cd-8189-4d7a-8c8f-a43b59bb4289 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:40.822328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2740b-27b6-4cd7-bd04-8de6d981c683 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:40.822509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c2740b-27b6-4cd7-bd04-8de6d981c683 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:40.823045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f88cd-8189-4d7a-8c8f-a43b59bb4289 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:40.823564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2f88cd-8189-4d7a-8c8f-a43b59bb4289 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:40.824407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f88cd-8189-4d7a-8c8f-a43b59bb4289 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:40.825048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f88cd-8189-4d7a-8c8f-a43b59bb4289 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:40.826923 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2f88cd-8189-4d7a-8c8f-a43b59bb4289 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:40.828644 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:40.831122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2740b-27b6-4cd7-bd04-8de6d981c683 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:40.832739 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c2740b-27b6-4cd7-bd04-8de6d981c683 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:40.841665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 267/532] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:40.842708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f88cd-8189-4d7a-8c8f-a43b59bb4289 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:40.849302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-16c2740b-27b6-4cd7-bd04-8de6d981c683 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:40.849302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:40.849302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:40.849302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:40.849302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:40.849302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:40.849302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:40.849302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:40.849302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:40.849302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:40.849302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:40.849302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:40.849302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:40.849302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:40.849302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:40.856969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:40.857790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:40.857790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:40.857790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:40.857790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:40.857790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:40.857790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:40.857790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:40.857790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:40.857790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 266/533] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:40.857790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2740b-27b6-4cd7-bd04-8de6d981c683 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:40.908607 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:40.968932 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2f88cd-8189-4d7a-8c8f-a43b59bb4289 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:40.974362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c2740b-27b6-4cd7-bd04-8de6d981c683 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:41.009810 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49c8b8-212f-4cfe-be15-66726cb1d33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:41.011144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49c8b8-212f-4cfe-be15-66726cb1d33d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:41.011614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49c8b8-212f-4cfe-be15-66726cb1d33d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:41.012336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd49c8b8-212f-4cfe-be15-66726cb1d33d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:41.016275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ac25e-b3d5-476d-8504-ff31fc00fd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:41.016970 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ac25e-b3d5-476d-8504-ff31fc00fd34 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:41.017369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ac25e-b3d5-476d-8504-ff31fc00fd34 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:41.018305 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ac25e-b3d5-476d-8504-ff31fc00fd34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:41.021827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49c8b8-212f-4cfe-be15-66726cb1d33d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:41.023454 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd49c8b8-212f-4cfe-be15-66726cb1d33d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:41.025643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ac25e-b3d5-476d-8504-ff31fc00fd34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:41.027259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ac25e-b3d5-476d-8504-ff31fc00fd34 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:41.036965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-cd49c8b8-212f-4cfe-be15-66726cb1d33d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:41.036965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:41.036965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:41.036965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:41.036965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:41.036965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:41.036965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:41.036965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:41.036965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:41.036965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:41.036965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:41.036965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:41.036965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:41.036965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:41.036965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:41.037780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:41.038593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:41.038593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:41.038593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:41.038593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:41.038593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:41.038593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:41.038593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:41.038593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:41.038987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6e0ac25e-b3d5-476d-8504-ff31fc00fd34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:41.038987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:41.038987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:41.038987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:41.038987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:41.038987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:41.038987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:41.038987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:41.038987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:41.038987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:41.038987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:41.038987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:41.038987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:41.038987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:41.038987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:41.039719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:41.040551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:41.040551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:41.040551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:41.040551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:41.040551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:41.040551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:41.040551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:41.040551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:41.042896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 268/534] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:41.043873 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49c8b8-212f-4cfe-be15-66726cb1d33d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:41.046984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 267/535] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:41.048796 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ac25e-b3d5-476d-8504-ff31fc00fd34 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:41.105122 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:41.156986 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd49c8b8-212f-4cfe-be15-66726cb1d33d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:41.160944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ac25e-b3d5-476d-8504-ff31fc00fd34 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:41.162545 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:41.197526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cbb46-7b24-4191-beb5-82ed162e3561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:41.198464 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cbb46-7b24-4191-beb5-82ed162e3561 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:41.198924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cbb46-7b24-4191-beb5-82ed162e3561 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:41.199442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4cbb46-7b24-4191-beb5-82ed162e3561 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:41.200466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cbb46-7b24-4191-beb5-82ed162e3561 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:41.201013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cbb46-7b24-4191-beb5-82ed162e3561 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:41.202707 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016fbcb3-f631-44f6-8b33-f1e415d9dc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:41.203074 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4cbb46-7b24-4191-beb5-82ed162e3561 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:41.203496 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fbcb3-f631-44f6-8b33-f1e415d9dc98 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:41.203917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fbcb3-f631-44f6-8b33-f1e415d9dc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:41.204296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016fbcb3-f631-44f6-8b33-f1e415d9dc98 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:41.205209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016fbcb3-f631-44f6-8b33-f1e415d9dc98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:41.205769 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016fbcb3-f631-44f6-8b33-f1e415d9dc98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:41.207397 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016fbcb3-f631-44f6-8b33-f1e415d9dc98 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:41.225622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 268/536] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:41.231025 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016fbcb3-f631-44f6-8b33-f1e415d9dc98 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:41.231170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 269/537] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:41.231170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cbb46-7b24-4191-beb5-82ed162e3561 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:41.343788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016fbcb3-f631-44f6-8b33-f1e415d9dc98 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:41.350981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4cbb46-7b24-4191-beb5-82ed162e3561 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:41.369477 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:41.389299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd9781-7a7e-4e41-b6b6-f0793f82d242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:41.390349 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd9781-7a7e-4e41-b6b6-f0793f82d242 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:41.391020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd9781-7a7e-4e41-b6b6-f0793f82d242 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:41.391385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd9781-7a7e-4e41-b6b6-f0793f82d242 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:41.392233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd9781-7a7e-4e41-b6b6-f0793f82d242 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:41.393356 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd9781-7a7e-4e41-b6b6-f0793f82d242 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:41.394898 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd9781-7a7e-4e41-b6b6-f0793f82d242 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:41.396199 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759b8a7-be37-45f5-baa3-73a1663c756a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:41.397164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759b8a7-be37-45f5-baa3-73a1663c756a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:41.397593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759b8a7-be37-45f5-baa3-73a1663c756a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:41.398048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759b8a7-be37-45f5-baa3-73a1663c756a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:41.399590 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759b8a7-be37-45f5-baa3-73a1663c756a None admin] RBAC: Adding query filter params (role.id=90322db354f94f03b32ec088922baf2e, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=e6dee005fc3d4fb593a1771e7851c20a) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:41.400153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759b8a7-be37-45f5-baa3-73a1663c756a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:41.401838 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759b8a7-be37-45f5-baa3-73a1663c756a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:41.409571 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 270/538] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:41 2022] GET /identity/v3/role_assignments?user.id=e6dee005fc3d4fb593a1771e7851c20a&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=90322db354f94f03b32ec088922baf2e => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:41.410074 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 269/539] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:41.412566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd9781-7a7e-4e41-b6b6-f0793f82d242 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:41.415560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759b8a7-be37-45f5-baa3-73a1663c756a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:41.464684 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:41.572995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759b8a7-be37-45f5-baa3-73a1663c756a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:41.616173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec88461-1742-4247-b42e-fce24d18c396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:41.616800 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec88461-1742-4247-b42e-fce24d18c396 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:41.617081 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec88461-1742-4247-b42e-fce24d18c396 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:41.617349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec88461-1742-4247-b42e-fce24d18c396 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:41.618085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec88461-1742-4247-b42e-fce24d18c396 None admin] RBAC: Adding query filter params (role.id=90322db354f94f03b32ec088922baf2e, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=ef7e0b772a284e7da4fae7d73f34d3de) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:41.618415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec88461-1742-4247-b42e-fce24d18c396 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:41.620051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec88461-1742-4247-b42e-fce24d18c396 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:41.629880 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd9781-7a7e-4e41-b6b6-f0793f82d242 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:41.633673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 271/540] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:41 2022] GET /identity/v3/role_assignments?user.id=ef7e0b772a284e7da4fae7d73f34d3de&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=90322db354f94f03b32ec088922baf2e => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:41.672202 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a3a7b-0b54-4ee8-b1c5-7ae63b651290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:41.672827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a3a7b-0b54-4ee8-b1c5-7ae63b651290 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:41.673105 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a3a7b-0b54-4ee8-b1c5-7ae63b651290 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:41.673379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a3a7b-0b54-4ee8-b1c5-7ae63b651290 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:41.674643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a3a7b-0b54-4ee8-b1c5-7ae63b651290 None admin] RBAC: Adding query filter params (role.id=90322db354f94f03b32ec088922baf2e, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=e6d98d51966f48d3a1b22f9b1c7963be) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:41.675026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a3a7b-0b54-4ee8-b1c5-7ae63b651290 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:41.676595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a3a7b-0b54-4ee8-b1c5-7ae63b651290 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:41.687719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 270/541] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:41 2022] GET /identity/v3/role_assignments?user.id=e6d98d51966f48d3a1b22f9b1c7963be&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=90322db354f94f03b32ec088922baf2e => generated 259 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:42.325081 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab549557-a7f5-4f70-8771-4a60265ffa59 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:42.325775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab549557-a7f5-4f70-8771-4a60265ffa59 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:42.326258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab549557-a7f5-4f70-8771-4a60265ffa59 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:42.328113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 272/542] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:42.334664 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ff9c8-e7b8-4e4b-b8bd-98f43c8c941c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:42.335130 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ff9c8-e7b8-4e4b-b8bd-98f43c8c941c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:42.335224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ff9c8-e7b8-4e4b-b8bd-98f43c8c941c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:42.360689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-437ff9c8-e7b8-4e4b-b8bd-98f43c8c941c None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:42.362469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-437ff9c8-e7b8-4e4b-b8bd-98f43c8c941c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:42.421447 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235a385-245f-481a-9a21-8f42fd4baf17 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:42.422063 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235a385-245f-481a-9a21-8f42fd4baf17 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:42.422537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b235a385-245f-481a-9a21-8f42fd4baf17 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:42.427747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 273/543] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:42.431666 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5fb77-ef17-40a0-8a48-01a45a5c2ea7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:42.432179 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5fb77-ef17-40a0-8a48-01a45a5c2ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:42.432654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c5fb77-ef17-40a0-8a48-01a45a5c2ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:42.466396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 271/544] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:42.467669 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e0c5fb77-ef17-40a0-8a48-01a45a5c2ea7 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:42.468895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e0c5fb77-ef17-40a0-8a48-01a45a5c2ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:42.524210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:42.570652 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 274/545] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:44:42.982476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ff9c8-e7b8-4e4b-b8bd-98f43c8c941c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:43.005685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:43.011114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5fb77-ef17-40a0-8a48-01a45a5c2ea7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:43.148523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c5fb77-ef17-40a0-8a48-01a45a5c2ea7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:43.171619 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ff9c8-e7b8-4e4b-b8bd-98f43c8c941c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:43.187265 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d14d6-0b99-4a0b-a315-aa1a3eb14d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:43.187936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d14d6-0b99-4a0b-a315-aa1a3eb14d28 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:43.190435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d14d6-0b99-4a0b-a315-aa1a3eb14d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:43.190717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d14d6-0b99-4a0b-a315-aa1a3eb14d28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:43.212331 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d14d6-0b99-4a0b-a315-aa1a3eb14d28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-843d14d6-0b99-4a0b-a315-aa1a3eb14d28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:43.217200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:43.218042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:43.218794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:43.218794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:43.218794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:43.218794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:43.218794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:43.218794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:43.218794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:43.218794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:43.218794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:43.218794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:43.218794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:43.218794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:43.218794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 275/546] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:44:43.229545 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d14d6-0b99-4a0b-a315-aa1a3eb14d28 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:43.243242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ad989-3594-4bf8-86eb-0a8bb26d93d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:43.243851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ad989-3594-4bf8-86eb-0a8bb26d93d2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:43.244134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ad989-3594-4bf8-86eb-0a8bb26d93d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:43.244408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ad989-3594-4bf8-86eb-0a8bb26d93d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:43.268677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ad989-3594-4bf8-86eb-0a8bb26d93d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-df9ad989-3594-4bf8-86eb-0a8bb26d93d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:43.273656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:43.274526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:43.275285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:43.275285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:43.275285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:43.275285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:43.275285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:43.275285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:43.275285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:43.275285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:43.275285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:43.275285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:43.275285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:43.275285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:43.275285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 272/547] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:43.284919 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ad989-3594-4bf8-86eb-0a8bb26d93d2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:43.296120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:43.361953 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d14d6-0b99-4a0b-a315-aa1a3eb14d28 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:43.398626 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07437b6-4d5a-4105-a871-010b72104b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:43.399677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07437b6-4d5a-4105-a871-010b72104b5b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:43.400190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07437b6-4d5a-4105-a871-010b72104b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:43.400664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07437b6-4d5a-4105-a871-010b72104b5b None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:43.401515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07437b6-4d5a-4105-a871-010b72104b5b None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:43.402009 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07437b6-4d5a-4105-a871-010b72104b5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:43.403626 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07437b6-4d5a-4105-a871-010b72104b5b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:43.412953 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 276/548] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:44:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:43.420506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07437b6-4d5a-4105-a871-010b72104b5b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:43.475705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:43.522343 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ad989-3594-4bf8-86eb-0a8bb26d93d2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:43.580215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:43.587639 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07437b6-4d5a-4105-a871-010b72104b5b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:43.595321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048b4d8-7bc2-446b-8b6d-5c72b6b6d683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:43.596545 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048b4d8-7bc2-446b-8b6d-5c72b6b6d683 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:43.597022 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048b4d8-7bc2-446b-8b6d-5c72b6b6d683 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:43.597420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d048b4d8-7bc2-446b-8b6d-5c72b6b6d683 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:43.598330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048b4d8-7bc2-446b-8b6d-5c72b6b6d683 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:43.598795 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048b4d8-7bc2-446b-8b6d-5c72b6b6d683 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:43.600369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d048b4d8-7bc2-446b-8b6d-5c72b6b6d683 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:43.621953 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 273/549] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:44:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:43.632604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dd2d9-0036-4e91-a84d-efad53c0d670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:43.633196 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dd2d9-0036-4e91-a84d-efad53c0d670 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:43.633477 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dd2d9-0036-4e91-a84d-efad53c0d670 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:43.633748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476dd2d9-0036-4e91-a84d-efad53c0d670 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:43.641419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048b4d8-7bc2-446b-8b6d-5c72b6b6d683 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:43.647264 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dd2d9-0036-4e91-a84d-efad53c0d670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:43.648775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476dd2d9-0036-4e91-a84d-efad53c0d670 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-476dd2d9-0036-4e91-a84d-efad53c0d670 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:43.664053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:43.665485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:43.666560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:43.666560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:43.666560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:43.669838 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 277/550] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:43.675303 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dd2d9-0036-4e91-a84d-efad53c0d670 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:43.855876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d048b4d8-7bc2-446b-8b6d-5c72b6b6d683 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:43.875419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476dd2d9-0036-4e91-a84d-efad53c0d670 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:43.910620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f09323-c7e2-448f-930b-6f03f8053f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:43.911731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f09323-c7e2-448f-930b-6f03f8053f16 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:43.912388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f09323-c7e2-448f-930b-6f03f8053f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:43.912854 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f09323-c7e2-448f-930b-6f03f8053f16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:43.920319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:43.922829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaf5c36-b644-4f38-95c6-d5bfe4723daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:43.923636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf5c36-b644-4f38-95c6-d5bfe4723daf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:43.924162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf5c36-b644-4f38-95c6-d5bfe4723daf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:43.924576 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f09323-c7e2-448f-930b-6f03f8053f16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:43.924961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaf5c36-b644-4f38-95c6-d5bfe4723daf None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:43.925759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf5c36-b644-4f38-95c6-d5bfe4723daf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:43.926289 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf5c36-b644-4f38-95c6-d5bfe4723daf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:43.926408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f09323-c7e2-448f-930b-6f03f8053f16 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:43.928180 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaf5c36-b644-4f38-95c6-d5bfe4723daf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:43.941180 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 278/551] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-83f09323-c7e2-448f-930b-6f03f8053f16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:43.941844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:43.942782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:43.943608 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:43.943608 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:43.943608 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:43.943940 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8354da-84b5-418f-b4c9-df0ece40ff32 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:43.943940 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8354da-84b5-418f-b4c9-df0ece40ff32 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:43.943940 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8354da-84b5-418f-b4c9-df0ece40ff32 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:43.945857 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 274/553] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:43.946926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f09323-c7e2-448f-930b-6f03f8053f16 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:43.951911 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 279/553] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:43.952639 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8354da-84b5-418f-b4c9-df0ece40ff32 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:43.964158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:44.075676 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f09323-c7e2-448f-930b-6f03f8053f16 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:44.086823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8354da-84b5-418f-b4c9-df0ece40ff32 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:44.129055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835a737-0d65-413b-99f6-d98afc87084d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:44.129425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b97750-7390-40a9-9cf2-9aa6fdd8ac2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:44.129688 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835a737-0d65-413b-99f6-d98afc87084d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.129967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835a737-0d65-413b-99f6-d98afc87084d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.130237 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835a737-0d65-413b-99f6-d98afc87084d None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.130895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835a737-0d65-413b-99f6-d98afc87084d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:44.131123 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b97750-7390-40a9-9cf2-9aa6fdd8ac2f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.131229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835a737-0d65-413b-99f6-d98afc87084d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:44.131729 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b97750-7390-40a9-9cf2-9aa6fdd8ac2f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.132211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b97750-7390-40a9-9cf2-9aa6fdd8ac2f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.132603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835a737-0d65-413b-99f6-d98afc87084d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:44.144306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b97750-7390-40a9-9cf2-9aa6fdd8ac2f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:44.146461 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b97750-7390-40a9-9cf2-9aa6fdd8ac2f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:44.151894 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 280/554] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:44.153275 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2af049-5114-4888-b948-2f1d50c96fa9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.153563 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2af049-5114-4888-b948-2f1d50c96fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.153837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2af049-5114-4888-b948-2f1d50c96fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-06b97750-7390-40a9-9cf2-9aa6fdd8ac2f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:44.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:44.162246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:44.163197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:44.163197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:44.163197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:44.163197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:44.163197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:44.163197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:44.163197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:44.163197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:44.163197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:44:44.163197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:44.165385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 275/555] 10.222.0.67 () {56 vars in 1093 bytes} [Mon Sep 26 17:44:43 2022] GET /identity/v3/users/nova => generated 82 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:44.166202 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b97750-7390-40a9-9cf2-9aa6fdd8ac2f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:44.179037 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-db2af049-5114-4888-b948-2f1d50c96fa9 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:44.180266 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-db2af049-5114-4888-b948-2f1d50c96fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:44.233624 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:44.251471 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:44.263690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 281/556] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:44.263690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2af049-5114-4888-b948-2f1d50c96fa9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:44.298257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b97750-7390-40a9-9cf2-9aa6fdd8ac2f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:44.343662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be71488-3fbf-49bb-b009-9df40a8b5ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:44.344439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be71488-3fbf-49bb-b009-9df40a8b5ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.344815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be71488-3fbf-49bb-b009-9df40a8b5ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.345179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be71488-3fbf-49bb-b009-9df40a8b5ba4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.356031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be71488-3fbf-49bb-b009-9df40a8b5ba4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:44.358066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be71488-3fbf-49bb-b009-9df40a8b5ba4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2be71488-3fbf-49bb-b009-9df40a8b5ba4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:44.373689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:44.375281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:44.376210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:44.376210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:44.376210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:44.376210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:44.376210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:44.376210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:44.376210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:44.376210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:44.376210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:44:44.376210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:44.377722 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2af049-5114-4888-b948-2f1d50c96fa9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:44.379842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 276/557] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:44:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:44.381302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f7975-a6b9-4503-bdff-de31bf462b8e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.381920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f7975-a6b9-4503-bdff-de31bf462b8e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.382191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39f7975-a6b9-4503-bdff-de31bf462b8e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.383565 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 277/558] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:44.384151 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f7975-a6b9-4503-bdff-de31bf462b8e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:44.423202 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea46afb-ac86-4374-a986-68164b09f9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:44.424402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea46afb-ac86-4374-a986-68164b09f9cb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.424681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea46afb-ac86-4374-a986-68164b09f9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.424951 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea46afb-ac86-4374-a986-68164b09f9cb None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.426144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea46afb-ac86-4374-a986-68164b09f9cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:44.426446 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea46afb-ac86-4374-a986-68164b09f9cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:44.428271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea46afb-ac86-4374-a986-68164b09f9cb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:44.452764 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 282/559] 10.222.0.67 () {56 vars in 1133 bytes} [Mon Sep 26 17:44:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:44.454160 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c34b3f-c08c-4136-b27a-5feb460ad61b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.454908 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c34b3f-c08c-4136-b27a-5feb460ad61b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.455367 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c34b3f-c08c-4136-b27a-5feb460ad61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.476750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b1c34b3f-c08c-4136-b27a-5feb460ad61b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:44.478455 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b1c34b3f-c08c-4136-b27a-5feb460ad61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:44.534200 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39f7975-a6b9-4503-bdff-de31bf462b8e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:44.536110 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:44.551956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:44.554109 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 283/560] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:44.555049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c34b3f-c08c-4136-b27a-5feb460ad61b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:44.583065 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1fba36-5df9-481c-a06a-86069b2ef3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:44.583757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fba36-5df9-481c-a06a-86069b2ef3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.584037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fba36-5df9-481c-a06a-86069b2ef3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.584308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1fba36-5df9-481c-a06a-86069b2ef3b9 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.585034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fba36-5df9-481c-a06a-86069b2ef3b9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:44.585459 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fba36-5df9-481c-a06a-86069b2ef3b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:44.586991 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1fba36-5df9-481c-a06a-86069b2ef3b9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:44.605515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 278/561] 10.222.0.67 () {56 vars in 1135 bytes} [Mon Sep 26 17:44:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:44.606715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e967e7-00d7-450f-b4af-b7b8c1f00fb8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.606997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e967e7-00d7-450f-b4af-b7b8c1f00fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.607268 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e967e7-00d7-450f-b4af-b7b8c1f00fb8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.608591 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 279/562] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:44.609149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e967e7-00d7-450f-b4af-b7b8c1f00fb8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:44.665284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c34b3f-c08c-4136-b27a-5feb460ad61b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:44.701121 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f5de8-3e68-4ed0-b5e3-7c1f65efff4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:44.701755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5de8-3e68-4ed0-b5e3-7c1f65efff4f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.702041 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5de8-3e68-4ed0-b5e3-7c1f65efff4f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.702314 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5de8-3e68-4ed0-b5e3-7c1f65efff4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.714590 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f5de8-3e68-4ed0-b5e3-7c1f65efff4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:44.716306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441f5de8-3e68-4ed0-b5e3-7c1f65efff4f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:44.723159 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e967e7-00d7-450f-b4af-b7b8c1f00fb8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-441f5de8-3e68-4ed0-b5e3-7c1f65efff4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:44.733049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:44.733965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:44.734812 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:44.734812 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:44.734812 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:44.734812 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 284/563] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:44.735520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244420ac-d0fb-4d26-ac03-8c2dd172aebd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.735898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244420ac-d0fb-4d26-ac03-8c2dd172aebd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.736013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244420ac-d0fb-4d26-ac03-8c2dd172aebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.756879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-244420ac-d0fb-4d26-ac03-8c2dd172aebd None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:44.758828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-244420ac-d0fb-4d26-ac03-8c2dd172aebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:44.762645 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7e928-880d-46b0-8daa-5a48268bd94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:44.763268 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7e928-880d-46b0-8daa-5a48268bd94f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.763546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7e928-880d-46b0-8daa-5a48268bd94f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.764096 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad7e928-880d-46b0-8daa-5a48268bd94f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.775021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad7e928-880d-46b0-8daa-5a48268bd94f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2ad7e928-880d-46b0-8daa-5a48268bd94f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:44.780873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:44.783038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:44.784169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:44.784169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:44.784169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:44.784169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:44.784169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:44.784169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:44.784169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:44.784169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:44.784169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:44.784169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:44.784169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:44.784169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:44.785900 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 280/564] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:44.786844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7e928-880d-46b0-8daa-5a48268bd94f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:44.793299 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:44.816037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:44.836738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 285/565] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:44.837578 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244420ac-d0fb-4d26-ac03-8c2dd172aebd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:44.891752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad7e928-880d-46b0-8daa-5a48268bd94f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:44.928238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08c4d1-fb38-4a44-9c25-a129d15a0bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:44.928768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08c4d1-fb38-4a44-9c25-a129d15a0bbb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.929051 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08c4d1-fb38-4a44-9c25-a129d15a0bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.929328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08c4d1-fb38-4a44-9c25-a129d15a0bbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.950173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244420ac-d0fb-4d26-ac03-8c2dd172aebd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:44.990269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564f1dd-455f-448e-9727-39e8b28ac894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:44.990885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564f1dd-455f-448e-9727-39e8b28ac894 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:44.991172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564f1dd-455f-448e-9727-39e8b28ac894 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:44.991443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564f1dd-455f-448e-9727-39e8b28ac894 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:44.992103 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564f1dd-455f-448e-9727-39e8b28ac894 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:44.992410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564f1dd-455f-448e-9727-39e8b28ac894 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:44.993769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564f1dd-455f-448e-9727-39e8b28ac894 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:45.009497 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 286/566] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:45.009497 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564f1dd-455f-448e-9727-39e8b28ac894 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.010759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08c4d1-fb38-4a44-9c25-a129d15a0bbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:45.012488 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08c4d1-fb38-4a44-9c25-a129d15a0bbb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:45.034485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:45.041664 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:45.109637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564f1dd-455f-448e-9727-39e8b28ac894 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:45.117147 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 281/567] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:45.117980 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08c4d1-fb38-4a44-9c25-a129d15a0bbb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.146362 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7aaa6-5749-4ae7-9ceb-0c78eee5d86d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:45.146968 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7aaa6-5749-4ae7-9ceb-0c78eee5d86d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.147248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7aaa6-5749-4ae7-9ceb-0c78eee5d86d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.147521 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7aaa6-5749-4ae7-9ceb-0c78eee5d86d None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.148204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7aaa6-5749-4ae7-9ceb-0c78eee5d86d None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:45.148510 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7aaa6-5749-4ae7-9ceb-0c78eee5d86d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:45.149871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7aaa6-5749-4ae7-9ceb-0c78eee5d86d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:45.157840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 287/568] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:45.159073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7aaa6-5749-4ae7-9ceb-0c78eee5d86d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.221504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08c4d1-fb38-4a44-9c25-a129d15a0bbb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:45.246823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:45.248298 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:45.255720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38baf4fa-77d2-4229-9f5e-397c9715b2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:45.256349 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38baf4fa-77d2-4229-9f5e-397c9715b2b8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.256629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38baf4fa-77d2-4229-9f5e-397c9715b2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.257222 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38baf4fa-77d2-4229-9f5e-397c9715b2b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.258781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7aaa6-5749-4ae7-9ceb-0c78eee5d86d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:45.267193 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38baf4fa-77d2-4229-9f5e-397c9715b2b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-38baf4fa-77d2-4229-9f5e-397c9715b2b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:45.272825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:45.273792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:45.274662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:45.274662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:45.274662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:45.274662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:45.274662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:45.274662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:45.274662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:45.274662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:45.274662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:45.274662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:45.274662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:45.274662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:45.274662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 282/569] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:45.274662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38baf4fa-77d2-4229-9f5e-397c9715b2b8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.296501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e09653-4662-4aa9-9925-c9cc6c4f51b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:45.297129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e09653-4662-4aa9-9925-c9cc6c4f51b5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.297408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e09653-4662-4aa9-9925-c9cc6c4f51b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.297665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e09653-4662-4aa9-9925-c9cc6c4f51b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.304234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e09653-4662-4aa9-9925-c9cc6c4f51b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:45.305666 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e09653-4662-4aa9-9925-c9cc6c4f51b5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:45.316866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e1e09653-4662-4aa9-9925-c9cc6c4f51b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:45.316866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:45.316866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:45.316866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:45.316866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:45.316866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:45.316866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:45.316866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:45.316866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:45.316866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:45.316866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:45.316866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:45.316866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:45.316866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:45.316866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:45.317738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:45.318617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:45.318617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:45.318617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:45.318617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:45.318617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:45.318617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:45.318617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:45.318617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:45.318617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 288/570] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:45.319932 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e09653-4662-4aa9-9925-c9cc6c4f51b5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.374738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38baf4fa-77d2-4229-9f5e-397c9715b2b8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:45.408477 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6d477-5d19-4c46-bf22-417b3003d31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:45.409196 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6d477-5d19-4c46-bf22-417b3003d31a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.409473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6d477-5d19-4c46-bf22-417b3003d31a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.409743 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6d477-5d19-4c46-bf22-417b3003d31a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.420252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e6d477-5d19-4c46-bf22-417b3003d31a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:45.420385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e09653-4662-4aa9-9925-c9cc6c4f51b5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c8e6d477-5d19-4c46-bf22-417b3003d31a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:45.425107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:45.426194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:45.427016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:45.427016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:45.427016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:45.427016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:45.427016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:45.427016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:45.427016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:45.427016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:45.427016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:45.427016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:45.427016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:45.427016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:45.427016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 283/571] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:45.427016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6d477-5d19-4c46-bf22-417b3003d31a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.454753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75a349-6fba-4cd7-9894-f13792104d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:45.456153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75a349-6fba-4cd7-9894-f13792104d47 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.456442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75a349-6fba-4cd7-9894-f13792104d47 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.456724 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75a349-6fba-4cd7-9894-f13792104d47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.467211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75a349-6fba-4cd7-9894-f13792104d47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:45.468677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75a349-6fba-4cd7-9894-f13792104d47 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:45.477100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:45.477561 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2c75a349-6fba-4cd7-9894-f13792104d47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:45.484125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:45.490325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:45.491192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:45.491192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:45.491192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:45.491192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 289/572] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:45.491192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75a349-6fba-4cd7-9894-f13792104d47 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.534738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6d477-5d19-4c46-bf22-417b3003d31a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:45.568506 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d6875-baaa-4e90-a581-79a695b82052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:45.569139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d6875-baaa-4e90-a581-79a695b82052 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.569465 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d6875-baaa-4e90-a581-79a695b82052 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.569741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4d6875-baaa-4e90-a581-79a695b82052 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.579542 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d6875-baaa-4e90-a581-79a695b82052 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:45.581885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4d6875-baaa-4e90-a581-79a695b82052 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0b4d6875-baaa-4e90-a581-79a695b82052 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:45.596455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:45.597510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:45.598429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:45.598429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:45.598429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:45.598429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 284/573] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:45.598429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d6875-baaa-4e90-a581-79a695b82052 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.639781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75a349-6fba-4cd7-9894-f13792104d47 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:45.672277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c00f1-b94a-4a29-b147-a9b6ede273e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:45.672878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c00f1-b94a-4a29-b147-a9b6ede273e6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.673153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c00f1-b94a-4a29-b147-a9b6ede273e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.673421 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c00f1-b94a-4a29-b147-a9b6ede273e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.674197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c00f1-b94a-4a29-b147-a9b6ede273e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:45.674497 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c00f1-b94a-4a29-b147-a9b6ede273e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:45.675889 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c00f1-b94a-4a29-b147-a9b6ede273e6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:45.683927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 290/574] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:45.684695 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c00f1-b94a-4a29-b147-a9b6ede273e6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.696335 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:45.699284 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4d6875-baaa-4e90-a581-79a695b82052 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:45.736487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04854be9-bf12-4498-bfd6-5df941efbe1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:45.737006 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04854be9-bf12-4498-bfd6-5df941efbe1b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.737281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04854be9-bf12-4498-bfd6-5df941efbe1b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.737549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04854be9-bf12-4498-bfd6-5df941efbe1b None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.738230 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04854be9-bf12-4498-bfd6-5df941efbe1b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:45.738530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04854be9-bf12-4498-bfd6-5df941efbe1b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:45.739977 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04854be9-bf12-4498-bfd6-5df941efbe1b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:45.748511 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:45.753046 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 285/575] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:45.753851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04854be9-bf12-4498-bfd6-5df941efbe1b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.794972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c00f1-b94a-4a29-b147-a9b6ede273e6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:45.828805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbede335-7137-4867-8743-c6f7b94f0770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:45.829408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbede335-7137-4867-8743-c6f7b94f0770 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.829683 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbede335-7137-4867-8743-c6f7b94f0770 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.830604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbede335-7137-4867-8743-c6f7b94f0770 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.831278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbede335-7137-4867-8743-c6f7b94f0770 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:45.831561 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbede335-7137-4867-8743-c6f7b94f0770 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:45.832942 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbede335-7137-4867-8743-c6f7b94f0770 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:45.840705 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 291/576] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:45.841469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbede335-7137-4867-8743-c6f7b94f0770 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.858269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04854be9-bf12-4498-bfd6-5df941efbe1b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:45.893237 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc4a47-df1a-43d8-b8d0-371787fd94f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:45.893805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4a47-df1a-43d8-b8d0-371787fd94f5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.894082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4a47-df1a-43d8-b8d0-371787fd94f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.894348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc4a47-df1a-43d8-b8d0-371787fd94f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.894972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc4a47-df1a-43d8-b8d0-371787fd94f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:45.895265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc4a47-df1a-43d8-b8d0-371787fd94f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:45.896700 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc4a47-df1a-43d8-b8d0-371787fd94f5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:45.907574 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 286/577] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:45.912247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc4a47-df1a-43d8-b8d0-371787fd94f5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:45.931106 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:45.947106 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbede335-7137-4867-8743-c6f7b94f0770 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:45.969765 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:45.984304 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c01051-3fea-4756-919a-b43768679071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:45.984921 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c01051-3fea-4756-919a-b43768679071 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:45.985198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c01051-3fea-4756-919a-b43768679071 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:45.985470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c01051-3fea-4756-919a-b43768679071 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:45.986705 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c01051-3fea-4756-919a-b43768679071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:45.987062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c01051-3fea-4756-919a-b43768679071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:45.988683 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c01051-3fea-4756-919a-b43768679071 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.001429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 292/578] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:46.002904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c01051-3fea-4756-919a-b43768679071 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.013238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc4a47-df1a-43d8-b8d0-371787fd94f5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.046190 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1ed4d-baca-45e4-9aaf-8f8792715218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.046822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1ed4d-baca-45e4-9aaf-8f8792715218 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.047104 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1ed4d-baca-45e4-9aaf-8f8792715218 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.047382 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d1ed4d-baca-45e4-9aaf-8f8792715218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.058802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1ed4d-baca-45e4-9aaf-8f8792715218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.060290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d1ed4d-baca-45e4-9aaf-8f8792715218 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-63d1ed4d-baca-45e4-9aaf-8f8792715218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:46.074823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:46.075797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:46.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:46.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:46.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:46.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 287/579] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:46.076869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1ed4d-baca-45e4-9aaf-8f8792715218 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.101405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c01051-3fea-4756-919a-b43768679071 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.135806 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46f735-a07e-43f9-865c-05ffa7590527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.136468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46f735-a07e-43f9-865c-05ffa7590527 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.136744 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46f735-a07e-43f9-865c-05ffa7590527 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.137012 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da46f735-a07e-43f9-865c-05ffa7590527 None admin] PATH_INFO: `/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/b9ed29a3f6be4d64bb3751f9e95eb56b/roles/90322db354f94f03b32ec088922baf2e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.148528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:46.151981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da46f735-a07e-43f9-865c-05ffa7590527 None admin] RBAC: Authorizing `identity:create_grant(project_id=71c96e5caeb44927bc21504fe54f0f98, user_id=b9ed29a3f6be4d64bb3751f9e95eb56b, role_id=90322db354f94f03b32ec088922baf2e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.153419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da46f735-a07e-43f9-865c-05ffa7590527 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.178098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d1ed4d-baca-45e4-9aaf-8f8792715218 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.211824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442f3fd-ce67-4e16-8869-6c2384c1816f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.212481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442f3fd-ce67-4e16-8869-6c2384c1816f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.212759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442f3fd-ce67-4e16-8869-6c2384c1816f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.213032 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442f3fd-ce67-4e16-8869-6c2384c1816f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.226122 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 293/580] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:44:46 2022] PUT /identity/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/b9ed29a3f6be4d64bb3751f9e95eb56b/roles/90322db354f94f03b32ec088922baf2e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:46.226122 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46f735-a07e-43f9-865c-05ffa7590527 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.229149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442f3fd-ce67-4e16-8869-6c2384c1816f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.229149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442f3fd-ce67-4e16-8869-6c2384c1816f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c442f3fd-ce67-4e16-8869-6c2384c1816f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:46.257204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:46.258252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:46.259137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:46.259137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:46.259137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:46.259137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 288/581] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:46.259137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442f3fd-ce67-4e16-8869-6c2384c1816f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.277029 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:46.354236 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da46f735-a07e-43f9-865c-05ffa7590527 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.376010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442f3fd-ce67-4e16-8869-6c2384c1816f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.393769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c748d8f-7360-4021-8bb2-053e5e8e1136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.395102 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c748d8f-7360-4021-8bb2-053e5e8e1136 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.395102 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c748d8f-7360-4021-8bb2-053e5e8e1136 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.395102 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c748d8f-7360-4021-8bb2-053e5e8e1136 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.403002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c748d8f-7360-4021-8bb2-053e5e8e1136 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.404475 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c748d8f-7360-4021-8bb2-053e5e8e1136 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.406898 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:46.416533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-3c748d8f-7360-4021-8bb2-053e5e8e1136 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:46.416533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:46.416533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:46.416533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:46.416533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:46.416533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:46.416533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:46.416533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:46.416533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:46.416533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:46.416533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:46.416533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:46.416533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:46.416533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:46.416533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:46.419015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:46.419909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:46.419909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:46.419909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:46.419909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:46.419909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:46.419909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:46.419909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:46.419909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:46.419909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 294/582] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:46.420397 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a78615-51ed-4bd1-bb4d-9aec5728979f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.420397 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a78615-51ed-4bd1-bb4d-9aec5728979f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.420549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c748d8f-7360-4021-8bb2-053e5e8e1136 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.420639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a78615-51ed-4bd1-bb4d-9aec5728979f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.420639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a78615-51ed-4bd1-bb4d-9aec5728979f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.428977 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a78615-51ed-4bd1-bb4d-9aec5728979f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.430559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a78615-51ed-4bd1-bb4d-9aec5728979f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-90a78615-51ed-4bd1-bb4d-9aec5728979f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:46.444272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:46.445408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:46.446374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:46.446374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:46.446374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:46.446374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:46.446374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:46.446374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:46.446374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:46.446374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:46.446374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 26 17:44:46.446374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:46.446374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 289/583] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:46.446939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a78615-51ed-4bd1-bb4d-9aec5728979f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.528880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:46.536222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c748d8f-7360-4021-8bb2-053e5e8e1136 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.553791 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a78615-51ed-4bd1-bb4d-9aec5728979f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.578710 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f144954-5aad-45d6-9cb9-337b6b57012b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.579633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f144954-5aad-45d6-9cb9-337b6b57012b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.580655 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f144954-5aad-45d6-9cb9-337b6b57012b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.580995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f144954-5aad-45d6-9cb9-337b6b57012b None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.581960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f144954-5aad-45d6-9cb9-337b6b57012b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:46.582711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f144954-5aad-45d6-9cb9-337b6b57012b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.584698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f144954-5aad-45d6-9cb9-337b6b57012b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.589853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36f439-b041-4e07-87c8-473f53aa6a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.590462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36f439-b041-4e07-87c8-473f53aa6a92 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.590740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36f439-b041-4e07-87c8-473f53aa6a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.591011 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36f439-b041-4e07-87c8-473f53aa6a92 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.591670 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36f439-b041-4e07-87c8-473f53aa6a92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:46.591980 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36f439-b041-4e07-87c8-473f53aa6a92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.593380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c36f439-b041-4e07-87c8-473f53aa6a92 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.602941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 295/584] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:46.602941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f144954-5aad-45d6-9cb9-337b6b57012b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.610780 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 290/585] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:46.610780 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36f439-b041-4e07-87c8-473f53aa6a92 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.655815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:46.715397 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c36f439-b041-4e07-87c8-473f53aa6a92 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.720071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f144954-5aad-45d6-9cb9-337b6b57012b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.752777 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e07f7-b059-497c-b98a-a9f59c9d1d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.753526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e07f7-b059-497c-b98a-a9f59c9d1d17 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.753924 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e07f7-b059-497c-b98a-a9f59c9d1d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.754289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82e07f7-b059-497c-b98a-a9f59c9d1d17 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.755125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e07f7-b059-497c-b98a-a9f59c9d1d17 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:46.755422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e07f7-b059-497c-b98a-a9f59c9d1d17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.756875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82e07f7-b059-497c-b98a-a9f59c9d1d17 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.757186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bc776-0fd0-46e0-92e1-cc1399c9ed84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.757789 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bc776-0fd0-46e0-92e1-cc1399c9ed84 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.758065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bc776-0fd0-46e0-92e1-cc1399c9ed84 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.758597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0bc776-0fd0-46e0-92e1-cc1399c9ed84 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.760154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0bc776-0fd0-46e0-92e1-cc1399c9ed84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:46.760470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0bc776-0fd0-46e0-92e1-cc1399c9ed84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.761870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0bc776-0fd0-46e0-92e1-cc1399c9ed84 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.775178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 296/586] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:46.775598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 291/587] 10.222.0.67 () {56 vars in 1137 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:46.776375 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bc776-0fd0-46e0-92e1-cc1399c9ed84 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.776535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e07f7-b059-497c-b98a-a9f59c9d1d17 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.797168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:46.894269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0bc776-0fd0-46e0-92e1-cc1399c9ed84 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.901244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82e07f7-b059-497c-b98a-a9f59c9d1d17 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:46.903614 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:46.930318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd62e1-40ef-4a4e-947e-44ff8bf76da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.931081 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd62e1-40ef-4a4e-947e-44ff8bf76da1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.931375 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd62e1-40ef-4a4e-947e-44ff8bf76da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.931740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd62e1-40ef-4a4e-947e-44ff8bf76da1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.939557 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd62e1-40ef-4a4e-947e-44ff8bf76da1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.941099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fd62e1-40ef-4a4e-947e-44ff8bf76da1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.942225 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747c78a-ff00-4be9-80b2-2c93e8ccb00e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:46.943132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747c78a-ff00-4be9-80b2-2c93e8ccb00e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:46.943414 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747c78a-ff00-4be9-80b2-2c93e8ccb00e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:46.944304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d747c78a-ff00-4be9-80b2-2c93e8ccb00e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:46.951910 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747c78a-ff00-4be9-80b2-2c93e8ccb00e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-30fd62e1-40ef-4a4e-947e-44ff8bf76da1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:46.952784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:46.953863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:46.955520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:46.955520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:46.955520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:46.955520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:46.955520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:46.955520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:46.955520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:46.955520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:46.955520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 17:44:46.955520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:46.956237 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d747c78a-ff00-4be9-80b2-2c93e8ccb00e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:46.956735 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 297/588] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:46.957239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd62e1-40ef-4a4e-947e-44ff8bf76da1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d747c78a-ff00-4be9-80b2-2c93e8ccb00e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:46.967944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:46.969226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:46.970408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:46.970408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:46.970408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:46.970408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:46.970408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:46.970408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:46.970408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:46.970408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:46.970408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:44:46.970408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:46.972224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 292/589] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:46.972848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747c78a-ff00-4be9-80b2-2c93e8ccb00e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.052153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:47.078314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d747c78a-ff00-4be9-80b2-2c93e8ccb00e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.081200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fd62e1-40ef-4a4e-947e-44ff8bf76da1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.113819 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214391ab-c66e-4bc2-a00e-a8a075ef0af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.114428 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214391ab-c66e-4bc2-a00e-a8a075ef0af9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.114706 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214391ab-c66e-4bc2-a00e-a8a075ef0af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.114977 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214391ab-c66e-4bc2-a00e-a8a075ef0af9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.116037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214391ab-c66e-4bc2-a00e-a8a075ef0af9 None admin] RBAC: Adding query filter params (role.id=90322db354f94f03b32ec088922baf2e, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=7ba12b94717a4a40b55635192840d3c3) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:47.116229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214391ab-c66e-4bc2-a00e-a8a075ef0af9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.117708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214391ab-c66e-4bc2-a00e-a8a075ef0af9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.121554 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae197d0-3b0c-476f-95f2-58f995d4aa0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.122206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae197d0-3b0c-476f-95f2-58f995d4aa0e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.122487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae197d0-3b0c-476f-95f2-58f995d4aa0e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.122752 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae197d0-3b0c-476f-95f2-58f995d4aa0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.125105 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 293/590] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/role_assignments?user.id=7ba12b94717a4a40b55635192840d3c3&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=90322db354f94f03b32ec088922baf2e => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:47.125978 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214391ab-c66e-4bc2-a00e-a8a075ef0af9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.158101 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae197d0-3b0c-476f-95f2-58f995d4aa0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.159717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae197d0-3b0c-476f-95f2-58f995d4aa0e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.176353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-3ae197d0-3b0c-476f-95f2-58f995d4aa0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:47.177769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:47.178830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:47.179914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:47.179914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:47.179914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:47.185967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 298/591] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:47.187085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae197d0-3b0c-476f-95f2-58f995d4aa0e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.272595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214391ab-c66e-4bc2-a00e-a8a075ef0af9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.313517 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430364c-69bc-4ad8-b68a-96d7d810ba6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.314048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430364c-69bc-4ad8-b68a-96d7d810ba6d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.314922 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430364c-69bc-4ad8-b68a-96d7d810ba6d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.315266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430364c-69bc-4ad8-b68a-96d7d810ba6d None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.316614 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430364c-69bc-4ad8-b68a-96d7d810ba6d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:47.317091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430364c-69bc-4ad8-b68a-96d7d810ba6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.319289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430364c-69bc-4ad8-b68a-96d7d810ba6d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.325883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae197d0-3b0c-476f-95f2-58f995d4aa0e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.342940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 294/592] 10.222.0.67 () {56 vars in 1139 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:47.343567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430364c-69bc-4ad8-b68a-96d7d810ba6d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.368756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:47.380815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b114754-fd60-443e-b433-593abd599c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.382278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b114754-fd60-443e-b433-593abd599c56 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.390986 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b114754-fd60-443e-b433-593abd599c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.390986 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b114754-fd60-443e-b433-593abd599c56 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.390986 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b114754-fd60-443e-b433-593abd599c56 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:47.390986 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b114754-fd60-443e-b433-593abd599c56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.390986 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b114754-fd60-443e-b433-593abd599c56 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.409961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 299/593] 10.222.0.67 () {56 vars in 1137 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:47.410795 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b114754-fd60-443e-b433-593abd599c56 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.456622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:47.478703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430364c-69bc-4ad8-b68a-96d7d810ba6d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.524535 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b114754-fd60-443e-b433-593abd599c56 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.526298 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d4a52-941c-4245-b409-3f68d7713023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.527889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d4a52-941c-4245-b409-3f68d7713023 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.528188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d4a52-941c-4245-b409-3f68d7713023 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.528463 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d4a52-941c-4245-b409-3f68d7713023 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.529121 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d4a52-941c-4245-b409-3f68d7713023 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:47.529423 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d4a52-941c-4245-b409-3f68d7713023 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.530791 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d4a52-941c-4245-b409-3f68d7713023 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.548605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 295/594] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:47.549924 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d4a52-941c-4245-b409-3f68d7713023 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.572537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7018e9-f332-4647-b88b-82c0e3b1d282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.573502 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7018e9-f332-4647-b88b-82c0e3b1d282 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.573994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7018e9-f332-4647-b88b-82c0e3b1d282 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.574967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7018e9-f332-4647-b88b-82c0e3b1d282 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.590457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7018e9-f332-4647-b88b-82c0e3b1d282 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.592309 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7018e9-f332-4647-b88b-82c0e3b1d282 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-fb7018e9-f332-4647-b88b-82c0e3b1d282 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:47.613588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:47.614547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:47.615384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:47.615384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:47.615384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:47.621959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 300/595] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:47.621959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7018e9-f332-4647-b88b-82c0e3b1d282 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.668779 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d4a52-941c-4245-b409-3f68d7713023 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.677775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:47.707618 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db4a5d-fec7-40ae-be49-ed727f3a93b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.708285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db4a5d-fec7-40ae-be49-ed727f3a93b7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.708564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db4a5d-fec7-40ae-be49-ed727f3a93b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.708838 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1db4a5d-fec7-40ae-be49-ed727f3a93b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.722492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db4a5d-fec7-40ae-be49-ed727f3a93b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.724167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1db4a5d-fec7-40ae-be49-ed727f3a93b7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.727303 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f1db4a5d-fec7-40ae-be49-ed727f3a93b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:47.741282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:47.744091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:47.744948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:47.744948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:47.744948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:47.744948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 296/596] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:47.744948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db4a5d-fec7-40ae-be49-ed727f3a93b7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.769016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7018e9-f332-4647-b88b-82c0e3b1d282 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.814346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42cf6f-aee0-430a-94ae-8bfbd3b7d59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.816167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42cf6f-aee0-430a-94ae-8bfbd3b7d59c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.817082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42cf6f-aee0-430a-94ae-8bfbd3b7d59c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.817796 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea42cf6f-aee0-430a-94ae-8bfbd3b7d59c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.827734 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42cf6f-aee0-430a-94ae-8bfbd3b7d59c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.829476 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea42cf6f-aee0-430a-94ae-8bfbd3b7d59c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.843704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ea42cf6f-aee0-430a-94ae-8bfbd3b7d59c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:47.843704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:47.843704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:47.843704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:47.843704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:47.843704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:47.843704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:47.843704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:47.843704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:47.843704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:47.843704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:47.843704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:47.843704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:47.843704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:47.843704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:47.844681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:47.846065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:47.846065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:47.846065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:47.846065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:47.846065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:47.846065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:47.846065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:47.846065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:47.850585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 301/597] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:47.852173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42cf6f-aee0-430a-94ae-8bfbd3b7d59c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.866949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1db4a5d-fec7-40ae-be49-ed727f3a93b7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:47.911509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a8260-8a99-4e9c-b8e5-3b2d4527c0be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:47.913131 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a8260-8a99-4e9c-b8e5-3b2d4527c0be None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:47.913612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a8260-8a99-4e9c-b8e5-3b2d4527c0be None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:47.914078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7a8260-8a99-4e9c-b8e5-3b2d4527c0be None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:47.915285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a8260-8a99-4e9c-b8e5-3b2d4527c0be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:47.915842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a8260-8a99-4e9c-b8e5-3b2d4527c0be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:47.917976 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7a8260-8a99-4e9c-b8e5-3b2d4527c0be None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:47.941971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 297/598] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:47.941971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a8260-8a99-4e9c-b8e5-3b2d4527c0be None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:47.988294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:47.995625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea42cf6f-aee0-430a-94ae-8bfbd3b7d59c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:48.018457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:48.042356 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2b1da-ecc3-4a79-957e-6b6d1a048cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:48.043102 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2b1da-ecc3-4a79-957e-6b6d1a048cbd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:48.043439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2b1da-ecc3-4a79-957e-6b6d1a048cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:48.043788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f2b1da-ecc3-4a79-957e-6b6d1a048cbd None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:48.044505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f2b1da-ecc3-4a79-957e-6b6d1a048cbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:48.044852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f2b1da-ecc3-4a79-957e-6b6d1a048cbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:48.046838 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f2b1da-ecc3-4a79-957e-6b6d1a048cbd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:48.063396 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 302/599] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:48.065341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2b1da-ecc3-4a79-957e-6b6d1a048cbd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:48.083628 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7a8260-8a99-4e9c-b8e5-3b2d4527c0be None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:48.141099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc33eb-dc21-441a-8531-de7d625fe121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:48.143261 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc33eb-dc21-441a-8531-de7d625fe121 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:48.144113 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc33eb-dc21-441a-8531-de7d625fe121 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:48.144878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97bc33eb-dc21-441a-8531-de7d625fe121 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:48.146321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc33eb-dc21-441a-8531-de7d625fe121 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:48.147332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc33eb-dc21-441a-8531-de7d625fe121 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:48.149352 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97bc33eb-dc21-441a-8531-de7d625fe121 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:48.169086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 298/600] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:48.170905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc33eb-dc21-441a-8531-de7d625fe121 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:48.225581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f2b1da-ecc3-4a79-957e-6b6d1a048cbd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:48.277943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94bfe68-4f3f-42b6-a41d-f145e90dfdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:48.278954 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bfe68-4f3f-42b6-a41d-f145e90dfdb9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:48.279476 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bfe68-4f3f-42b6-a41d-f145e90dfdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:48.279988 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bfe68-4f3f-42b6-a41d-f145e90dfdb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:48.290021 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94bfe68-4f3f-42b6-a41d-f145e90dfdb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:48.292156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94bfe68-4f3f-42b6-a41d-f145e90dfdb9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:48.309251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f94bfe68-4f3f-42b6-a41d-f145e90dfdb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:48.309251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:48.309251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:48.309251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:48.309251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:48.309251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:48.309251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:48.309251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:48.309251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:48.309251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:48.309251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:48.309251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:48.309251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:48.309251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:48.309251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:48.310178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:48.311018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:48.311018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:48.311018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:48.311018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:48.311018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:48.311018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:48.311018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:48.311018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:48.313421 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97bc33eb-dc21-441a-8531-de7d625fe121 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:48.313652 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 303/601] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:48.315937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94bfe68-4f3f-42b6-a41d-f145e90dfdb9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:48.341758 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:48.344118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:48.372399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d3472-557f-45a4-a9a5-d460e4e7fd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:48.373436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d3472-557f-45a4-a9a5-d460e4e7fd2e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:48.373748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d3472-557f-45a4-a9a5-d460e4e7fd2e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:48.374024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d3472-557f-45a4-a9a5-d460e4e7fd2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:48.385674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d3472-557f-45a4-a9a5-d460e4e7fd2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:48.388083 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d3472-557f-45a4-a9a5-d460e4e7fd2e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:48.405389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8b9d3472-557f-45a4-a9a5-d460e4e7fd2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:48.405389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:48.405389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:48.405389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:48.405389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:48.405389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:48.405389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:48.405389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:48.405389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:48.405389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:48.405389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:48.405389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:48.405389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:48.405389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:48.405389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:48.406329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:48.407165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:48.407165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:48.407165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:48.407165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:48.407165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:48.407165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:48.407165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:48.407165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:48.407165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 299/602] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:48.408395 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d3472-557f-45a4-a9a5-d460e4e7fd2e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:48.476591 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94bfe68-4f3f-42b6-a41d-f145e90dfdb9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:48.525133 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a0406a-7379-4d18-938e-9ebda6d53769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:48.525824 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0406a-7379-4d18-938e-9ebda6d53769 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:48.526153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0406a-7379-4d18-938e-9ebda6d53769 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:48.526360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a0406a-7379-4d18-938e-9ebda6d53769 None admin] PATH_INFO: `/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/e6dee005fc3d4fb593a1771e7851c20a/roles/90322db354f94f03b32ec088922baf2e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:48.546984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0406a-7379-4d18-938e-9ebda6d53769 None admin] RBAC: Authorizing `identity:create_grant(project_id=71c96e5caeb44927bc21504fe54f0f98, user_id=e6dee005fc3d4fb593a1771e7851c20a, role_id=90322db354f94f03b32ec088922baf2e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:48.548707 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a0406a-7379-4d18-938e-9ebda6d53769 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:48.557188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d3472-557f-45a4-a9a5-d460e4e7fd2e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:48.595076 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9dbe39-d64a-4771-bb3a-9383cb75bc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:48.596590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dbe39-d64a-4771-bb3a-9383cb75bc9a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:48.597053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dbe39-d64a-4771-bb3a-9383cb75bc9a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:48.597354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dbe39-d64a-4771-bb3a-9383cb75bc9a None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:48.598693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9dbe39-d64a-4771-bb3a-9383cb75bc9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:48.599074 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9dbe39-d64a-4771-bb3a-9383cb75bc9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:48.601206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9dbe39-d64a-4771-bb3a-9383cb75bc9a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:48.614961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 300/603] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:48.616020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9dbe39-d64a-4771-bb3a-9383cb75bc9a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:48.647676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 304/604] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:44:48 2022] PUT /identity/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/e6dee005fc3d4fb593a1771e7851c20a/roles/90322db354f94f03b32ec088922baf2e => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:48.647676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047c1fb-6864-41ef-976c-1c86c5b41f3c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:48.647676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047c1fb-6864-41ef-976c-1c86c5b41f3c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:48.647676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2047c1fb-6864-41ef-976c-1c86c5b41f3c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:48.647676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 305/605] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:48.647676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047c1fb-6864-41ef-976c-1c86c5b41f3c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:48.672030 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:48.761165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:48.768166 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9dbe39-d64a-4771-bb3a-9383cb75bc9a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:48.816549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2047c1fb-6864-41ef-976c-1c86c5b41f3c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:48.823337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a5657-2c3f-4ed5-ba03-19a84a5d8a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:48.824096 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a5657-2c3f-4ed5-ba03-19a84a5d8a12 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:48.824445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a5657-2c3f-4ed5-ba03-19a84a5d8a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:48.824794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a5657-2c3f-4ed5-ba03-19a84a5d8a12 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:48.825596 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a5657-2c3f-4ed5-ba03-19a84a5d8a12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:48.825972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a5657-2c3f-4ed5-ba03-19a84a5d8a12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:48.827595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a5657-2c3f-4ed5-ba03-19a84a5d8a12 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:48.846145 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 301/606] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:48.847747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de74bfe-d82a-48d1-b34e-0f2efe2a063e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:48.848113 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de74bfe-d82a-48d1-b34e-0f2efe2a063e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:48.848462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de74bfe-d82a-48d1-b34e-0f2efe2a063e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:48.871580 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6de74bfe-d82a-48d1-b34e-0f2efe2a063e None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:48.872226 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76823f-2cef-4b6b-ad9e-5590e196f67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:48.872975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76823f-2cef-4b6b-ad9e-5590e196f67d None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:48.873260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76823f-2cef-4b6b-ad9e-5590e196f67d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:48.873401 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6de74bfe-d82a-48d1-b34e-0f2efe2a063e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:48.873534 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c76823f-2cef-4b6b-ad9e-5590e196f67d None admin] PATH_INFO: `/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/ef7e0b772a284e7da4fae7d73f34d3de/roles/90322db354f94f03b32ec088922baf2e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:48.898834 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76823f-2cef-4b6b-ad9e-5590e196f67d None admin] RBAC: Authorizing `identity:create_grant(project_id=71c96e5caeb44927bc21504fe54f0f98, user_id=ef7e0b772a284e7da4fae7d73f34d3de, role_id=90322db354f94f03b32ec088922baf2e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:48.900406 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c76823f-2cef-4b6b-ad9e-5590e196f67d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:48.967884 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 306/607] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:44:48 2022] PUT /identity/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/ef7e0b772a284e7da4fae7d73f34d3de/roles/90322db354f94f03b32ec088922baf2e => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:48.968877 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76823f-2cef-4b6b-ad9e-5590e196f67d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:48.977284 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 302/608] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:49.141800 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c76823f-2cef-4b6b-ad9e-5590e196f67d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:49.168389 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:49.194330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42f37b-dab3-4e32-a590-1d1306ec8d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:49.195024 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42f37b-dab3-4e32-a590-1d1306ec8d0e None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:49.195316 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42f37b-dab3-4e32-a590-1d1306ec8d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:49.195597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad42f37b-dab3-4e32-a590-1d1306ec8d0e None admin] PATH_INFO: `/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/e6d98d51966f48d3a1b22f9b1c7963be/roles/90322db354f94f03b32ec088922baf2e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:49.214202 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42f37b-dab3-4e32-a590-1d1306ec8d0e None admin] RBAC: Authorizing `identity:create_grant(project_id=71c96e5caeb44927bc21504fe54f0f98, user_id=e6d98d51966f48d3a1b22f9b1c7963be, role_id=90322db354f94f03b32ec088922baf2e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:49.215985 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad42f37b-dab3-4e32-a590-1d1306ec8d0e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:49.328572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de74bfe-d82a-48d1-b34e-0f2efe2a063e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:49.337426 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 307/609] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:44:48 2022] PUT /identity/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/e6d98d51966f48d3a1b22f9b1c7963be/roles/90322db354f94f03b32ec088922baf2e => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:49.362552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:49.475163 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de74bfe-d82a-48d1-b34e-0f2efe2a063e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:49.529952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cbdcff-b536-4e31-ae43-4ec72d603590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:49.530612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbdcff-b536-4e31-ae43-4ec72d603590 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:49.530887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbdcff-b536-4e31-ae43-4ec72d603590 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:49.531154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cbdcff-b536-4e31-ae43-4ec72d603590 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:49.548730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cbdcff-b536-4e31-ae43-4ec72d603590 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-47cbdcff-b536-4e31-ae43-4ec72d603590 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:49.553979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:49.555071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:49.555958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:49.555958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:49.555958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:49.555958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:49.555958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:49.555958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:49.555958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:49.555958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:49.555958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:49.555958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:49.555958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:49.555958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:49.555958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 303/610] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:49.562919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42f37b-dab3-4e32-a590-1d1306ec8d0e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:49.604554 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862b63d-cb3a-4ca7-8701-6599a123ecc6 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:49.604894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862b63d-cb3a-4ca7-8701-6599a123ecc6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:49.605107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862b63d-cb3a-4ca7-8701-6599a123ecc6 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:49.606736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 304/611] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:49.620609 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54eda72-8876-4755-bf2f-eadfc78c72d0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:49.621585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54eda72-8876-4755-bf2f-eadfc78c72d0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:49.622101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54eda72-8876-4755-bf2f-eadfc78c72d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:49.648643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e54eda72-8876-4755-bf2f-eadfc78c72d0 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:49.650984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e54eda72-8876-4755-bf2f-eadfc78c72d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:49.737699 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad42f37b-dab3-4e32-a590-1d1306ec8d0e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:49.759392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 305/612] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:49.783956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee29203-59af-4739-ae89-f0adbccc0b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:49.784625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee29203-59af-4739-ae89-f0adbccc0b94 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:49.784917 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee29203-59af-4739-ae89-f0adbccc0b94 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:49.785195 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee29203-59af-4739-ae89-f0adbccc0b94 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:49.785867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee29203-59af-4739-ae89-f0adbccc0b94 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:49.786170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee29203-59af-4739-ae89-f0adbccc0b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:49.787541 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee29203-59af-4739-ae89-f0adbccc0b94 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:49.796457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 308/613] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:44:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:49.802424 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54eda72-8876-4755-bf2f-eadfc78c72d0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:49.823287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:49.917122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54eda72-8876-4755-bf2f-eadfc78c72d0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:49.963840 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5f6ff-45d8-4db7-9b52-238ca77baf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:49.964482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5f6ff-45d8-4db7-9b52-238ca77baf90 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:49.964763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5f6ff-45d8-4db7-9b52-238ca77baf90 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:49.965039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5f6ff-45d8-4db7-9b52-238ca77baf90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:49.982761 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5f6ff-45d8-4db7-9b52-238ca77baf90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:49.984327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc5f6ff-45d8-4db7-9b52-238ca77baf90 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4dc5f6ff-45d8-4db7-9b52-238ca77baf90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:50.004086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:50.005019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:50.005891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:50.005891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:50.005891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:50.007446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 306/614] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:50.012117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee29203-59af-4739-ae89-f0adbccc0b94 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:50.055016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:50.164274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee29203-59af-4739-ae89-f0adbccc0b94 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:50.203919 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5f6ff-45d8-4db7-9b52-238ca77baf90 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:50.221245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8175d-2633-4ec9-9329-d7083475b40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:50.221868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8175d-2633-4ec9-9329-d7083475b40f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:50.223717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8175d-2633-4ec9-9329-d7083475b40f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:50.223939 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa8175d-2633-4ec9-9329-d7083475b40f None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:50.224597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8175d-2633-4ec9-9329-d7083475b40f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:50.225836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8175d-2633-4ec9-9329-d7083475b40f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:50.227246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa8175d-2633-4ec9-9329-d7083475b40f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:50.253548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 309/615] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:50.258220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8175d-2633-4ec9-9329-d7083475b40f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:50.330017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:50.396513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5f6ff-45d8-4db7-9b52-238ca77baf90 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:50.415219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:50.429786 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa8175d-2633-4ec9-9329-d7083475b40f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:50.467593 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b468b-1a74-4f63-aa4c-adf4eceeb01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:50.469060 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b468b-1a74-4f63-aa4c-adf4eceeb01f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:50.469548 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b468b-1a74-4f63-aa4c-adf4eceeb01f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:50.471074 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b468b-1a74-4f63-aa4c-adf4eceeb01f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:50.473852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8b00c-4c2d-4c67-b93b-e8b1d6b27413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:50.476008 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8b00c-4c2d-4c67-b93b-e8b1d6b27413 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:50.476503 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8b00c-4c2d-4c67-b93b-e8b1d6b27413 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:50.476962 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8b00c-4c2d-4c67-b93b-e8b1d6b27413 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:50.491215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8b00c-4c2d-4c67-b93b-e8b1d6b27413 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:50.494836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d8b00c-4c2d-4c67-b93b-e8b1d6b27413 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:50.496153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1b468b-1a74-4f63-aa4c-adf4eceeb01f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1a1b468b-1a74-4f63-aa4c-adf4eceeb01f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:50.501975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:50.503021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:50.503913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:50.503913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:50.503913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:50.503913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:50.503913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:50.503913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:50.503913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:50.503913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:50.503913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:50.503913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:50.503913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:50.503913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:50.512021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 307/616] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a3d8b00c-4c2d-4c67-b93b-e8b1d6b27413 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:50.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:50.520055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:50.521156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:50.521156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:50.521156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:50.521156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:50.521156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:50.521156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:50.521156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:50.521156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:50.521156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:44:50.521156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:50.523673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b468b-1a74-4f63-aa4c-adf4eceeb01f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:50.528360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 310/617] 10.222.0.67 () {56 vars in 1093 bytes} [Mon Sep 26 17:44:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:44:50.533833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8b00c-4c2d-4c67-b93b-e8b1d6b27413 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:50.671015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b468b-1a74-4f63-aa4c-adf4eceeb01f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:50.725658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8b00c-4c2d-4c67-b93b-e8b1d6b27413 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:50.732618 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:50.749257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa21338-572c-4881-9075-c464416a39d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:50.751905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa21338-572c-4881-9075-c464416a39d8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:50.751905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa21338-572c-4881-9075-c464416a39d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:50.751905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa21338-572c-4881-9075-c464416a39d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:50.751905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa21338-572c-4881-9075-c464416a39d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:50.751905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa21338-572c-4881-9075-c464416a39d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:50.753079 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa21338-572c-4881-9075-c464416a39d8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:50.761833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f5c80-9187-4774-ae26-a9b471daf278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:50.762509 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f5c80-9187-4774-ae26-a9b471daf278 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:50.762732 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f5c80-9187-4774-ae26-a9b471daf278 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:50.763002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f5c80-9187-4774-ae26-a9b471daf278 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:50.763759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f5c80-9187-4774-ae26-a9b471daf278 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:50.764347 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f5c80-9187-4774-ae26-a9b471daf278 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:50.765777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47f5c80-9187-4774-ae26-a9b471daf278 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:50.772728 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 308/618] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:44:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:50.778896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa21338-572c-4881-9075-c464416a39d8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:50.786981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 311/619] 10.222.0.67 () {56 vars in 1133 bytes} [Mon Sep 26 17:44:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:50.792863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f5c80-9187-4774-ae26-a9b471daf278 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:50.812322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:50.920857 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa21338-572c-4881-9075-c464416a39d8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:50.958835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c76d6-d448-40e5-ab58-8e42c1f25058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:50.959759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c76d6-d448-40e5-ab58-8e42c1f25058 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:50.960178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c76d6-d448-40e5-ab58-8e42c1f25058 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:50.960604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22c76d6-d448-40e5-ab58-8e42c1f25058 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:50.972636 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c76d6-d448-40e5-ab58-8e42c1f25058 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:50.974367 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22c76d6-d448-40e5-ab58-8e42c1f25058 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d22c76d6-d448-40e5-ab58-8e42c1f25058 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:51.001757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:51.002952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:51.004042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:51.004042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:51.004042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:51.005919 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 309/620] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:51.011521 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c76d6-d448-40e5-ab58-8e42c1f25058 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:51.024432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f5c80-9187-4774-ae26-a9b471daf278 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:51.059303 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:51.122510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22c76d6-d448-40e5-ab58-8e42c1f25058 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:51.132351 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f9891-ec9f-4849-9e04-4086c14a6601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:51.133105 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f9891-ec9f-4849-9e04-4086c14a6601 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.133484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f9891-ec9f-4849-9e04-4086c14a6601 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.133849 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f9891-ec9f-4849-9e04-4086c14a6601 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.164018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb9b41-69e6-4434-9e7b-1244387d315a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:51.165066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb9b41-69e6-4434-9e7b-1244387d315a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.165539 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb9b41-69e6-4434-9e7b-1244387d315a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.166022 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb9b41-69e6-4434-9e7b-1244387d315a None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.167215 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb9b41-69e6-4434-9e7b-1244387d315a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:51.167967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb9b41-69e6-4434-9e7b-1244387d315a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:51.169703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bb9b41-69e6-4434-9e7b-1244387d315a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:51.186925 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 310/621] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:51.193409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb9b41-69e6-4434-9e7b-1244387d315a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:51.288381 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f9891-ec9f-4849-9e04-4086c14a6601 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:51.289764 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53f9891-ec9f-4849-9e04-4086c14a6601 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:51.306539 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:51.308092 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb9b41-69e6-4434-9e7b-1244387d315a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:51.310063 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:51.346046 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d463eaf4-d846-4005-947a-66dac772cd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:51.347014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463eaf4-d846-4005-947a-66dac772cd06 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.347467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463eaf4-d846-4005-947a-66dac772cd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.347943 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d463eaf4-d846-4005-947a-66dac772cd06 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.355576 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463eaf4-d846-4005-947a-66dac772cd06 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:51.357325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d463eaf4-d846-4005-947a-66dac772cd06 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d463eaf4-d846-4005-947a-66dac772cd06 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:51.372321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:51.373291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:51.374212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:51.374212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:51.374212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:51.374212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:51.374212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:51.374212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:51.374212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:51.374212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:51.374212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:44:51.374212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:51.376617 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 311/622] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:44:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:51.378225 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d052b-82fe-4cf5-9dca-d266918a9e91 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.378661 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d052b-82fe-4cf5-9dca-d266918a9e91 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.379072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19d052b-82fe-4cf5-9dca-d266918a9e91 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.380888 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 312/623] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:51.389148 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3084390-9cba-4b61-b64b-680e0ff28758 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.389148 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3084390-9cba-4b61-b64b-680e0ff28758 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.389148 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3084390-9cba-4b61-b64b-680e0ff28758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.403716 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 312/624] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:51.404571 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f9891-ec9f-4849-9e04-4086c14a6601 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:51.407430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c3084390-9cba-4b61-b64b-680e0ff28758 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:51.409768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c3084390-9cba-4b61-b64b-680e0ff28758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:51.489075 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 313/625] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:51.490015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3084390-9cba-4b61-b64b-680e0ff28758 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:51.509684 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f9891-ec9f-4849-9e04-4086c14a6601 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:51.518147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:51.545872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09941d3-8163-4ac5-8d6c-2ddf2484091f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:51.546409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09941d3-8163-4ac5-8d6c-2ddf2484091f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.546698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09941d3-8163-4ac5-8d6c-2ddf2484091f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.546974 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09941d3-8163-4ac5-8d6c-2ddf2484091f None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.547728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09941d3-8163-4ac5-8d6c-2ddf2484091f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:51.548037 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09941d3-8163-4ac5-8d6c-2ddf2484091f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:51.549524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09941d3-8163-4ac5-8d6c-2ddf2484091f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:51.568126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 313/626] 10.222.0.67 () {56 vars in 1135 bytes} [Mon Sep 26 17:44:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:51.576305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09941d3-8163-4ac5-8d6c-2ddf2484091f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:51.682803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:51.702432 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3084390-9cba-4b61-b64b-680e0ff28758 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:51.736180 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09941d3-8163-4ac5-8d6c-2ddf2484091f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:51.775225 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e929c79b-fdc4-4f96-bbe8-37a4d6b11f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:51.775888 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929c79b-fdc4-4f96-bbe8-37a4d6b11f90 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.776169 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929c79b-fdc4-4f96-bbe8-37a4d6b11f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.777294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e929c79b-fdc4-4f96-bbe8-37a4d6b11f90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.786212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9933da30-04d9-4346-ad0b-43c3436cefd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:51.787543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933da30-04d9-4346-ad0b-43c3436cefd0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.788084 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933da30-04d9-4346-ad0b-43c3436cefd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.788656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9933da30-04d9-4346-ad0b-43c3436cefd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.790625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929c79b-fdc4-4f96-bbe8-37a4d6b11f90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:51.792330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e929c79b-fdc4-4f96-bbe8-37a4d6b11f90 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:51.800843 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933da30-04d9-4346-ad0b-43c3436cefd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:51.802713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9933da30-04d9-4346-ad0b-43c3436cefd0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e929c79b-fdc4-4f96-bbe8-37a4d6b11f90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:51.808295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:51.809232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:51.810079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:51.810079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:51.810079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:51.810079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 314/627] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:51.811006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6a0ef-2a63-48e8-95db-e31beb8a687b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.811294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6a0ef-2a63-48e8-95db-e31beb8a687b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.811570 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6a0ef-2a63-48e8-95db-e31beb8a687b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.813293 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 315/628] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:51.817875 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6a0ef-2a63-48e8-95db-e31beb8a687b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9933da30-04d9-4346-ad0b-43c3436cefd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:51.824612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:51.825626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:51.826459 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:51.826459 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:51.826459 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:51.828255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 314/629] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:51.829780 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5de6b-f596-4144-b6e9-b175807f91b5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.830339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5de6b-f596-4144-b6e9-b175807f91b5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.830903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5de6b-f596-4144-b6e9-b175807f91b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.838705 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:51.851570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b7a5de6b-f596-4144-b6e9-b175807f91b5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:51.858156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b7a5de6b-f596-4144-b6e9-b175807f91b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:51.925227 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 315/630] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:51.926384 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5de6b-f596-4144-b6e9-b175807f91b5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:51.937207 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6a0ef-2a63-48e8-95db-e31beb8a687b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:51.973904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fb6e0-a27c-4c9a-acfc-b92b7db12bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:51.974607 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fb6e0-a27c-4c9a-acfc-b92b7db12bab None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:51.974956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fb6e0-a27c-4c9a-acfc-b92b7db12bab None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:51.979896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fb6e0-a27c-4c9a-acfc-b92b7db12bab None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:51.980844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679fb6e0-a27c-4c9a-acfc-b92b7db12bab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:51.983112 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679fb6e0-a27c-4c9a-acfc-b92b7db12bab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:51.987374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679fb6e0-a27c-4c9a-acfc-b92b7db12bab None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:52.006071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 316/631] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:52.007231 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fb6e0-a27c-4c9a-acfc-b92b7db12bab None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.022846 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:52.056915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5de6b-f596-4144-b6e9-b175807f91b5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.097382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:52.106856 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c257bf5-e4b6-4090-9c6e-b3e251b1870d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.107571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c257bf5-e4b6-4090-9c6e-b3e251b1870d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.109883 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c257bf5-e4b6-4090-9c6e-b3e251b1870d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.109883 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c257bf5-e4b6-4090-9c6e-b3e251b1870d None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.109883 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c257bf5-e4b6-4090-9c6e-b3e251b1870d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:52.109883 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c257bf5-e4b6-4090-9c6e-b3e251b1870d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:52.111710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c257bf5-e4b6-4090-9c6e-b3e251b1870d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:52.123072 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fb6e0-a27c-4c9a-acfc-b92b7db12bab None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.131577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 316/632] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:52.131577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c257bf5-e4b6-4090-9c6e-b3e251b1870d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.171601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9948e-45b5-4ec3-8796-d70f8184a62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.172279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9948e-45b5-4ec3-8796-d70f8184a62c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.172562 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9948e-45b5-4ec3-8796-d70f8184a62c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.172837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc9948e-45b5-4ec3-8796-d70f8184a62c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.184827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc9948e-45b5-4ec3-8796-d70f8184a62c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ccc9948e-45b5-4ec3-8796-d70f8184a62c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:52.190354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:52.191278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:52.192162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:52.192162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:52.192162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:52.192162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:52.192162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:52.192162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:52.192162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:52.192162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:52.192162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:52.192162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:52.192162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:52.192162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:52.195777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 317/633] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:52.196907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9948e-45b5-4ec3-8796-d70f8184a62c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.246885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c257bf5-e4b6-4090-9c6e-b3e251b1870d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.280908 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118b5160-dc2a-4f30-a7f8-7098e08db11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.281524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b5160-dc2a-4f30-a7f8-7098e08db11d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.281799 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b5160-dc2a-4f30-a7f8-7098e08db11d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.282064 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118b5160-dc2a-4f30-a7f8-7098e08db11d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.284066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:52.288933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b5160-dc2a-4f30-a7f8-7098e08db11d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:52.290369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118b5160-dc2a-4f30-a7f8-7098e08db11d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:52.297933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc9948e-45b5-4ec3-8796-d70f8184a62c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.302068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-118b5160-dc2a-4f30-a7f8-7098e08db11d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:52.302068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:52.302068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:52.302068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:52.302068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:52.302068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:52.302068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:52.302068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:52.302068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:52.302068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:52.302068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:52.302068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:52.302068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:52.302068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:52.302068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:52.303149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:52.304487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:52.304487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:52.304487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:52.304487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:52.304487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:52.304487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:52.304487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:52.304487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:52.304487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 317/634] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:52.305062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dc12a-db3c-431f-a03b-9d06bb3d377e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.305062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dc12a-db3c-431f-a03b-9d06bb3d377e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.305223 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261dc12a-db3c-431f-a03b-9d06bb3d377e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.306431 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 318/635] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:52.307165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261dc12a-db3c-431f-a03b-9d06bb3d377e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.337766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1a8df-08cc-4f6e-8755-fd7834135ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.338388 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1a8df-08cc-4f6e-8755-fd7834135ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.338664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1a8df-08cc-4f6e-8755-fd7834135ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.338934 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1a8df-08cc-4f6e-8755-fd7834135ea6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.341567 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:52.346360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1a8df-08cc-4f6e-8755-fd7834135ea6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:52.347814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1a8df-08cc-4f6e-8755-fd7834135ea6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:52.358879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-57a1a8df-08cc-4f6e-8755-fd7834135ea6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:52.358879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:52.358879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:52.358879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:52.358879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:52.358879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:52.358879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:52.358879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:52.358879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:52.358879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:52.358879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:52.358879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:52.358879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:52.358879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:52.358879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:52.359913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:52.360750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:52.360750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:52.360750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:52.360750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:52.360750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:52.360750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:52.360750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:52.360750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:52.360750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 318/636] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:52.360750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1a8df-08cc-4f6e-8755-fd7834135ea6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.416264 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261dc12a-db3c-431f-a03b-9d06bb3d377e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.449371 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8bf449-adba-4417-bcbd-03134d67db53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.449964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bf449-adba-4417-bcbd-03134d67db53 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.450239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bf449-adba-4417-bcbd-03134d67db53 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.450509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8bf449-adba-4417-bcbd-03134d67db53 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.451146 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bf449-adba-4417-bcbd-03134d67db53 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:52.451443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bf449-adba-4417-bcbd-03134d67db53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:52.452835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8bf449-adba-4417-bcbd-03134d67db53 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:52.461056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 319/637] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:52.461400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1a8df-08cc-4f6e-8755-fd7834135ea6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.462189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a89c45-6ac2-4d38-b70e-5f8cdf1d75de None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.462471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a89c45-6ac2-4d38-b70e-5f8cdf1d75de None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.462747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a89c45-6ac2-4d38-b70e-5f8cdf1d75de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.480574 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-33a89c45-6ac2-4d38-b70e-5f8cdf1d75de None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:52.481775 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-33a89c45-6ac2-4d38-b70e-5f8cdf1d75de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:52.496171 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf4fae-a314-48f1-845d-aea27906dcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.496803 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf4fae-a314-48f1-845d-aea27906dcc9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.497085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf4fae-a314-48f1-845d-aea27906dcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.497357 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf4fae-a314-48f1-845d-aea27906dcc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.508425 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cf4fae-a314-48f1-845d-aea27906dcc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c0cf4fae-a314-48f1-845d-aea27906dcc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:52.513604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:52.515258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:52.516158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:52.516158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:52.516158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:52.516158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:52.516158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:52.516158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:52.516158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:52.516158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:52.516158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:52.516158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:52.516158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:52.516158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:52.516158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 319/638] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:52.516158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf4fae-a314-48f1-845d-aea27906dcc9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.535742 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:52.551959 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 320/639] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:52.552770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a89c45-6ac2-4d38-b70e-5f8cdf1d75de None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.589287 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:52.634515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cf4fae-a314-48f1-845d-aea27906dcc9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.658871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a89c45-6ac2-4d38-b70e-5f8cdf1d75de None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.675383 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774023e0-bbdb-40e2-b7fd-bfac5a71f46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.676184 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774023e0-bbdb-40e2-b7fd-bfac5a71f46e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.676493 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774023e0-bbdb-40e2-b7fd-bfac5a71f46e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.676841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774023e0-bbdb-40e2-b7fd-bfac5a71f46e None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.677572 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774023e0-bbdb-40e2-b7fd-bfac5a71f46e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:52.677804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774023e0-bbdb-40e2-b7fd-bfac5a71f46e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:52.679216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774023e0-bbdb-40e2-b7fd-bfac5a71f46e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:52.692873 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 320/640] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:52.693652 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774023e0-bbdb-40e2-b7fd-bfac5a71f46e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.698973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdec43-c826-4250-9b94-f7d2e81de9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.699589 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdec43-c826-4250-9b94-f7d2e81de9ee None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.699901 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdec43-c826-4250-9b94-f7d2e81de9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.700172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdec43-c826-4250-9b94-f7d2e81de9ee None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.700860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdec43-c826-4250-9b94-f7d2e81de9ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:52.701166 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdec43-c826-4250-9b94-f7d2e81de9ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:52.702592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cdec43-c826-4250-9b94-f7d2e81de9ee None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:52.719674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 321/641] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:52.719674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdec43-c826-4250-9b94-f7d2e81de9ee None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.773274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:52.804658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774023e0-bbdb-40e2-b7fd-bfac5a71f46e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.822047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:52.836480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cdec43-c826-4250-9b94-f7d2e81de9ee None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.843587 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9192e-7dac-4c5a-a090-00b09c890cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.844207 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9192e-7dac-4c5a-a090-00b09c890cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.844495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9192e-7dac-4c5a-a090-00b09c890cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.844752 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb9192e-7dac-4c5a-a090-00b09c890cb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.856065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9192e-7dac-4c5a-a090-00b09c890cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:52.857492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb9192e-7dac-4c5a-a090-00b09c890cb7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:52.870916 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbe6b7-d0b2-4d6f-baf6-f289f7ecf69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:52.871525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbe6b7-d0b2-4d6f-baf6-f289f7ecf69b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:52.871837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbe6b7-d0b2-4d6f-baf6-f289f7ecf69b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6eb9192e-7dac-4c5a-a090-00b09c890cb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:52.871948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:52.872972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:52.873839 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:52.873839 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:52.873839 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:52.874018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbe6b7-d0b2-4d6f-baf6-f289f7ecf69b None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:52.874018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbe6b7-d0b2-4d6f-baf6-f289f7ecf69b None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:52.874018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbe6b7-d0b2-4d6f-baf6-f289f7ecf69b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:52.874209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 321/642] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:52.874209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9192e-7dac-4c5a-a090-00b09c890cb7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.874763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbe6b7-d0b2-4d6f-baf6-f289f7ecf69b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:52.883574 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 322/643] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:52.884426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbe6b7-d0b2-4d6f-baf6-f289f7ecf69b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:52.974404 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb9192e-7dac-4c5a-a090-00b09c890cb7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.984533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbe6b7-d0b2-4d6f-baf6-f289f7ecf69b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:52.994156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:53.014891 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369016e4-5b98-4dbd-83b8-c94435bbf8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.015555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369016e4-5b98-4dbd-83b8-c94435bbf8f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.015863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369016e4-5b98-4dbd-83b8-c94435bbf8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.016142 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369016e4-5b98-4dbd-83b8-c94435bbf8f3 None admin] PATH_INFO: `/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/7ba12b94717a4a40b55635192840d3c3/roles/90322db354f94f03b32ec088922baf2e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.020268 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0ce81-62cb-4fd9-9cb5-f9aa642643d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.020886 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0ce81-62cb-4fd9-9cb5-f9aa642643d9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.021360 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0ce81-62cb-4fd9-9cb5-f9aa642643d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.021801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f0ce81-62cb-4fd9-9cb5-f9aa642643d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.022801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0ce81-62cb-4fd9-9cb5-f9aa642643d9 None admin] RBAC: Adding query filter params (role.id=90322db354f94f03b32ec088922baf2e, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=b9ed29a3f6be4d64bb3751f9e95eb56b) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:53.023114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0ce81-62cb-4fd9-9cb5-f9aa642643d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.024586 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f0ce81-62cb-4fd9-9cb5-f9aa642643d9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.030247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 323/644] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:52 2022] GET /identity/v3/role_assignments?user.id=b9ed29a3f6be4d64bb3751f9e95eb56b&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=90322db354f94f03b32ec088922baf2e => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:53.030507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369016e4-5b98-4dbd-83b8-c94435bbf8f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=71c96e5caeb44927bc21504fe54f0f98, user_id=7ba12b94717a4a40b55635192840d3c3, role_id=90322db354f94f03b32ec088922baf2e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.031023 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0ce81-62cb-4fd9-9cb5-f9aa642643d9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.031975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369016e4-5b98-4dbd-83b8-c94435bbf8f3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.112148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 322/645] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:44:52 2022] PUT /identity/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/7ba12b94717a4a40b55635192840d3c3/roles/90322db354f94f03b32ec088922baf2e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:44:53.119634 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369016e4-5b98-4dbd-83b8-c94435bbf8f3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.151024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f0ce81-62cb-4fd9-9cb5-f9aa642643d9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.177997 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:53.213020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f9351-d5b1-42a9-bd6b-d444e72d6ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.213800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f9351-d5b1-42a9-bd6b-d444e72d6ca5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.214079 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f9351-d5b1-42a9-bd6b-d444e72d6ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.214353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870f9351-d5b1-42a9-bd6b-d444e72d6ca5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.228920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870f9351-d5b1-42a9-bd6b-d444e72d6ca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-870f9351-d5b1-42a9-bd6b-d444e72d6ca5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:53.235549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:53.237325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:53.238329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:53.238329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:53.238329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:53.238329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:53.238329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:53.238329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:53.238329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:53.238329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:53.238329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:53.238329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:53.238329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:53.238329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:53.238329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 324/646] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:53.239054 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f9351-d5b1-42a9-bd6b-d444e72d6ca5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.273604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:53.297368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369016e4-5b98-4dbd-83b8-c94435bbf8f3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.347218 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09306da3-218e-4dfd-8972-d58a025355e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.347865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09306da3-218e-4dfd-8972-d58a025355e1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.348150 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09306da3-218e-4dfd-8972-d58a025355e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.348422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09306da3-218e-4dfd-8972-d58a025355e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.349063 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09306da3-218e-4dfd-8972-d58a025355e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:53.349361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09306da3-218e-4dfd-8972-d58a025355e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.351411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09306da3-218e-4dfd-8972-d58a025355e1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.366006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 323/647] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:53.367307 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09306da3-218e-4dfd-8972-d58a025355e1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.370511 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870f9351-d5b1-42a9-bd6b-d444e72d6ca5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.408762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d6e37-7d48-422f-a18e-04ed904a0c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.410564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d6e37-7d48-422f-a18e-04ed904a0c02 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.411056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d6e37-7d48-422f-a18e-04ed904a0c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.411556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d6e37-7d48-422f-a18e-04ed904a0c02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.426245 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d6e37-7d48-422f-a18e-04ed904a0c02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.428585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d6e37-7d48-422f-a18e-04ed904a0c02 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-406d6e37-7d48-422f-a18e-04ed904a0c02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:53.450406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:53.451389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:53.453016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:53.453016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:53.453016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:53.456339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 325/648] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:53.457347 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:53.462241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d6e37-7d48-422f-a18e-04ed904a0c02 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.485697 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09306da3-218e-4dfd-8972-d58a025355e1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.526263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ef185-754f-4c9d-94a1-86775f28f0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.527838 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ef185-754f-4c9d-94a1-86775f28f0f0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.528640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ef185-754f-4c9d-94a1-86775f28f0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.529344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ef185-754f-4c9d-94a1-86775f28f0f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.530201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ef185-754f-4c9d-94a1-86775f28f0f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:53.530697 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ef185-754f-4c9d-94a1-86775f28f0f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.532292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4ef185-754f-4c9d-94a1-86775f28f0f0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.543727 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 324/649] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:53.545252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ef185-754f-4c9d-94a1-86775f28f0f0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.553218 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:53.570113 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d6e37-7d48-422f-a18e-04ed904a0c02 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.606258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5a95b-9834-451e-8039-f1e7bc012775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.606996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5a95b-9834-451e-8039-f1e7bc012775 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.607371 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5a95b-9834-451e-8039-f1e7bc012775 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.607745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b5a95b-9834-451e-8039-f1e7bc012775 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.616210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b5a95b-9834-451e-8039-f1e7bc012775 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.617794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b5a95b-9834-451e-8039-f1e7bc012775 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-75b5a95b-9834-451e-8039-f1e7bc012775 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:53.630340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:53.631307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:53.632243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:53.632243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:53.632243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:53.632243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:53.632243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:53.632243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:53.632243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:53.632243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:53.632243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 26 17:44:53.632243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:53.632243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 326/650] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:53.632823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5a95b-9834-451e-8039-f1e7bc012775 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.672114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ef185-754f-4c9d-94a1-86775f28f0f0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.713374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683d572-6084-4939-844d-cf4be1599cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.714258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683d572-6084-4939-844d-cf4be1599cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.714686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683d572-6084-4939-844d-cf4be1599cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.715098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4683d572-6084-4939-844d-cf4be1599cc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.715924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4683d572-6084-4939-844d-cf4be1599cc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:53.716389 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4683d572-6084-4939-844d-cf4be1599cc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.717892 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4683d572-6084-4939-844d-cf4be1599cc4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.731993 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:53.735835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 325/651] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:53.738484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683d572-6084-4939-844d-cf4be1599cc4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.749628 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b5a95b-9834-451e-8039-f1e7bc012775 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.795529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5b887-b27d-417d-a343-ad34738f3ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.796561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5b887-b27d-417d-a343-ad34738f3ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.797021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5b887-b27d-417d-a343-ad34738f3ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.797494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f5b887-b27d-417d-a343-ad34738f3ef7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.813641 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5b887-b27d-417d-a343-ad34738f3ef7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.815992 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f5b887-b27d-417d-a343-ad34738f3ef7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.818807 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-37f5b887-b27d-417d-a343-ad34738f3ef7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:53.833448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:53.836257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:53.837546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:53.837546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:53.837546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:53.837546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 327/652] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:53.837546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5b887-b27d-417d-a343-ad34738f3ef7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.855831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4683d572-6084-4939-844d-cf4be1599cc4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.894615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7c1dd-cb61-4569-94d8-9151fa9f97a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:53.895261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7c1dd-cb61-4569-94d8-9151fa9f97a7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:53.895543 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7c1dd-cb61-4569-94d8-9151fa9f97a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:53.895846 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7c1dd-cb61-4569-94d8-9151fa9f97a7 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:53.896733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7c1dd-cb61-4569-94d8-9151fa9f97a7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:53.897091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7c1dd-cb61-4569-94d8-9151fa9f97a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:53.898515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7c1dd-cb61-4569-94d8-9151fa9f97a7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:53.923465 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 326/653] 10.222.0.67 () {56 vars in 1137 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:53.924858 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7c1dd-cb61-4569-94d8-9151fa9f97a7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:53.950852 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f5b887-b27d-417d-a343-ad34738f3ef7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:53.999805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:54.001297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4494fb2-2817-405a-ae02-b2c38da757f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:54.002550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4494fb2-2817-405a-ae02-b2c38da757f7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.002984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4494fb2-2817-405a-ae02-b2c38da757f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.003407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4494fb2-2817-405a-ae02-b2c38da757f7 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.013173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4494fb2-2817-405a-ae02-b2c38da757f7 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:54.015944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4494fb2-2817-405a-ae02-b2c38da757f7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d4494fb2-2817-405a-ae02-b2c38da757f7 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:54.030873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:54.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:54.033516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:54.033516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:54.033516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:54.033516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:54.033516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:54.033516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:54.033516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:54.033516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:54.033516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 26 17:44:54.033516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:54.036052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 328/654] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/users/neutron => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:54.037420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4494fb2-2817-405a-ae02-b2c38da757f7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:54.055393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7c1dd-cb61-4569-94d8-9151fa9f97a7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:54.098902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608182ed-82d7-47c9-a247-e5bbb350f2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:54.099530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608182ed-82d7-47c9-a247-e5bbb350f2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.099852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608182ed-82d7-47c9-a247-e5bbb350f2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.100219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608182ed-82d7-47c9-a247-e5bbb350f2a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.100780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608182ed-82d7-47c9-a247-e5bbb350f2a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:54.101065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608182ed-82d7-47c9-a247-e5bbb350f2a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:54.102717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608182ed-82d7-47c9-a247-e5bbb350f2a2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:54.103931 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:54.119898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 327/655] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:54.121066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608182ed-82d7-47c9-a247-e5bbb350f2a2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:54.158485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4494fb2-2817-405a-ae02-b2c38da757f7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:54.195507 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07596d2d-dfe8-4e1b-a5aa-125cfe7130cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:54.196545 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07596d2d-dfe8-4e1b-a5aa-125cfe7130cf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.197027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07596d2d-dfe8-4e1b-a5aa-125cfe7130cf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.197521 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07596d2d-dfe8-4e1b-a5aa-125cfe7130cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.246512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608182ed-82d7-47c9-a247-e5bbb350f2a2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:54.266548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:54.270101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07596d2d-dfe8-4e1b-a5aa-125cfe7130cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:54.271833 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07596d2d-dfe8-4e1b-a5aa-125cfe7130cf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:54.285294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09482b-99de-4f0d-9c9a-d3a1992b7137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:54.287363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09482b-99de-4f0d-9c9a-d3a1992b7137 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.287363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09482b-99de-4f0d-9c9a-d3a1992b7137 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.287363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d09482b-99de-4f0d-9c9a-d3a1992b7137 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.287363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09482b-99de-4f0d-9c9a-d3a1992b7137 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:54.287363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09482b-99de-4f0d-9c9a-d3a1992b7137 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:54.289142 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d09482b-99de-4f0d-9c9a-d3a1992b7137 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:54.315755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 328/656] 10.222.0.67 () {56 vars in 1139 bytes} [Mon Sep 26 17:44:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:54.316990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09482b-99de-4f0d-9c9a-d3a1992b7137 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:54.334733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:54.383727 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 329/657] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:54.384919 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07596d2d-dfe8-4e1b-a5aa-125cfe7130cf None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:54.438984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d09482b-99de-4f0d-9c9a-d3a1992b7137 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:54.475569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ce2be-699b-4b2f-9915-2f9eb653cbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:54.476242 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ce2be-699b-4b2f-9915-2f9eb653cbe1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.476526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ce2be-699b-4b2f-9915-2f9eb653cbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.476802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ce2be-699b-4b2f-9915-2f9eb653cbe1 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.484405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ce2be-699b-4b2f-9915-2f9eb653cbe1 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:54.486393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ce2be-699b-4b2f-9915-2f9eb653cbe1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-065ce2be-699b-4b2f-9915-2f9eb653cbe1 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:54.498933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:54.499918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:54.500823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:54.500823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:54.500823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:54.500823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:54.500823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:54.500823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:54.500823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:54.500823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:54.500823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:44:54.500823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:54.505322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 329/658] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:54.506230 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ce2be-699b-4b2f-9915-2f9eb653cbe1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:54.514867 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07596d2d-dfe8-4e1b-a5aa-125cfe7130cf None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:54.550275 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:54.556408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00e9ba-2c9b-47e9-ae12-98aceec551d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:54.557137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00e9ba-2c9b-47e9-ae12-98aceec551d3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.557529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00e9ba-2c9b-47e9-ae12-98aceec551d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.557902 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc00e9ba-2c9b-47e9-ae12-98aceec551d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.585754 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:54.631152 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00e9ba-2c9b-47e9-ae12-98aceec551d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:54.632845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc00e9ba-2c9b-47e9-ae12-98aceec551d3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:54.638642 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ce2be-699b-4b2f-9915-2f9eb653cbe1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:54.675034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac8821-92c7-4c90-92c7-359d6ae7ef9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:54.675844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac8821-92c7-4c90-92c7-359d6ae7ef9e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.676218 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac8821-92c7-4c90-92c7-359d6ae7ef9e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeac8821-92c7-4c90-92c7-359d6ae7ef9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.688531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac8821-92c7-4c90-92c7-359d6ae7ef9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:54.690818 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeac8821-92c7-4c90-92c7-359d6ae7ef9e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-eeac8821-92c7-4c90-92c7-359d6ae7ef9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:54.706274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:54.707214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:54.708107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:54.708107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:54.708107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:54.712230 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 330/659] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:54.713081 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac8821-92c7-4c90-92c7-359d6ae7ef9e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:54.753199 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 330/660] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:54.754467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00e9ba-2c9b-47e9-ae12-98aceec551d3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:54.811062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:54.817076 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:54.824131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeac8821-92c7-4c90-92c7-359d6ae7ef9e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:54.861278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88929cbe-c335-48fb-8983-9aeb690846a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:54.862131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88929cbe-c335-48fb-8983-9aeb690846a8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.862419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88929cbe-c335-48fb-8983-9aeb690846a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.862698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88929cbe-c335-48fb-8983-9aeb690846a8 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.863432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88929cbe-c335-48fb-8983-9aeb690846a8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:54.863765 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88929cbe-c335-48fb-8983-9aeb690846a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:54.864939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc00e9ba-2c9b-47e9-ae12-98aceec551d3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:54.865384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88929cbe-c335-48fb-8983-9aeb690846a8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:54.888757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 331/661] 10.222.0.67 () {56 vars in 1137 bytes} [Mon Sep 26 17:44:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:54.889918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88929cbe-c335-48fb-8983-9aeb690846a8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:54.901701 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10017e-5b3e-403b-b791-9d1d29b87d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:54.902667 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10017e-5b3e-403b-b791-9d1d29b87d2a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:54.903143 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10017e-5b3e-403b-b791-9d1d29b87d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:54.903694 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f10017e-5b3e-403b-b791-9d1d29b87d2a None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:54.904543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10017e-5b3e-403b-b791-9d1d29b87d2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:54.905129 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10017e-5b3e-403b-b791-9d1d29b87d2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:54.906803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f10017e-5b3e-403b-b791-9d1d29b87d2a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:54.920379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 331/662] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:54.921593 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10017e-5b3e-403b-b791-9d1d29b87d2a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:55.014267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88929cbe-c335-48fb-8983-9aeb690846a8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:55.033503 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f10017e-5b3e-403b-b791-9d1d29b87d2a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:55.051678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:55.053969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271ade8-b720-46e5-a8bc-1064d56ae460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:55.054619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271ade8-b720-46e5-a8bc-1064d56ae460 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.054899 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271ade8-b720-46e5-a8bc-1064d56ae460 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.055172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271ade8-b720-46e5-a8bc-1064d56ae460 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.066277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271ade8-b720-46e5-a8bc-1064d56ae460 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:55.067834 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271ade8-b720-46e5-a8bc-1064d56ae460 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:55.072729 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e376f7-666e-4854-8a50-bd12a611a001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:55.073589 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e376f7-666e-4854-8a50-bd12a611a001 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.074087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e376f7-666e-4854-8a50-bd12a611a001 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.074568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e376f7-666e-4854-8a50-bd12a611a001 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.076693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a271ade8-b720-46e5-a8bc-1064d56ae460 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:55.087453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:55.088448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:55.089295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:55.089295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:55.089295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:55.093531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 332/663] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:55.094384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271ade8-b720-46e5-a8bc-1064d56ae460 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:55.153605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e376f7-666e-4854-8a50-bd12a611a001 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:55.155501 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e376f7-666e-4854-8a50-bd12a611a001 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:55.207585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271ade8-b720-46e5-a8bc-1064d56ae460 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:55.245279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da4bca-6f9e-4e68-86cd-14eec931b811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:55.245843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da4bca-6f9e-4e68-86cd-14eec931b811 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.246127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da4bca-6f9e-4e68-86cd-14eec931b811 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.246406 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09da4bca-6f9e-4e68-86cd-14eec931b811 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.254613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da4bca-6f9e-4e68-86cd-14eec931b811 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:55.256235 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09da4bca-6f9e-4e68-86cd-14eec931b811 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:55.264603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 332/664] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:55.265675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e376f7-666e-4854-8a50-bd12a611a001 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:55.274603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-09da4bca-6f9e-4e68-86cd-14eec931b811 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:55.274603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:55.274603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:55.274603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:55.274603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:55.274603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:55.274603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:55.274603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:55.274603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:55.274603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:55.274603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:55.274603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:55.274603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:55.274603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:55.274603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:55.275496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:55.276405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:55.276405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:55.276405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:55.276405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:55.276405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:55.276405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:55.276405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:55.276405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:55.279843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 333/665] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:55.281791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da4bca-6f9e-4e68-86cd-14eec931b811 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:55.288152 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:55.362781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:55.386450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e376f7-666e-4854-8a50-bd12a611a001 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:55.413341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09da4bca-6f9e-4e68-86cd-14eec931b811 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:55.425717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb701318-8d0d-443a-9296-fdc039a6514a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:55.426592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb701318-8d0d-443a-9296-fdc039a6514a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.426868 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb701318-8d0d-443a-9296-fdc039a6514a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.427135 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb701318-8d0d-443a-9296-fdc039a6514a None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.427801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb701318-8d0d-443a-9296-fdc039a6514a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:55.428101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb701318-8d0d-443a-9296-fdc039a6514a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:55.429464 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb701318-8d0d-443a-9296-fdc039a6514a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:55.441534 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 333/666] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:55.446026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb701318-8d0d-443a-9296-fdc039a6514a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:55.450994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d785fec3-1fc5-43fa-a739-eb2e553380c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:55.451694 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785fec3-1fc5-43fa-a739-eb2e553380c2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.451971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785fec3-1fc5-43fa-a739-eb2e553380c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.452250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d785fec3-1fc5-43fa-a739-eb2e553380c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.463974 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785fec3-1fc5-43fa-a739-eb2e553380c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:55.465420 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d785fec3-1fc5-43fa-a739-eb2e553380c2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-d785fec3-1fc5-43fa-a739-eb2e553380c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:55.480117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:55.481245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:55.482306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:55.482306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:55.482306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:55.485593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 334/667] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:55.487482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73fec8-ed9e-446a-a8d2-e67366567f42 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.487795 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73fec8-ed9e-446a-a8d2-e67366567f42 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.488075 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb73fec8-ed9e-446a-a8d2-e67366567f42 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.489381 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 335/668] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:55.490135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73fec8-ed9e-446a-a8d2-e67366567f42 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:55.528138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:55.551549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb701318-8d0d-443a-9296-fdc039a6514a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:55.590334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab81c1a-3f3b-476a-ae63-6ff927c1b0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:55.590936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab81c1a-3f3b-476a-ae63-6ff927c1b0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.591211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab81c1a-3f3b-476a-ae63-6ff927c1b0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.591488 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab81c1a-3f3b-476a-ae63-6ff927c1b0c3 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.592224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab81c1a-3f3b-476a-ae63-6ff927c1b0c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:55.592524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab81c1a-3f3b-476a-ae63-6ff927c1b0c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:55.593052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb73fec8-ed9e-446a-a8d2-e67366567f42 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:55.593967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab81c1a-3f3b-476a-ae63-6ff927c1b0c3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:55.604361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:55.605839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 334/669] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:55.606504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab81c1a-3f3b-476a-ae63-6ff927c1b0c3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:55.628618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab63dc6-cfe5-4aae-b19a-6e00d1248965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:55.629270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab63dc6-cfe5-4aae-b19a-6e00d1248965 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.629810 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab63dc6-cfe5-4aae-b19a-6e00d1248965 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.630089 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab63dc6-cfe5-4aae-b19a-6e00d1248965 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.636609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab63dc6-cfe5-4aae-b19a-6e00d1248965 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:55.638023 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab63dc6-cfe5-4aae-b19a-6e00d1248965 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:55.650220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-7ab63dc6-cfe5-4aae-b19a-6e00d1248965 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:55.650220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:55.650220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:55.650220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:55.650220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:55.650220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:55.650220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:55.650220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:55.650220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:55.650220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:55.650220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:55.650220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:55.650220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:55.650220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:55.650220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:55.651073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:55.652344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:55.652344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:55.652344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:55.652344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:55.652344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:55.652344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:55.652344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:55.652344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:55.652344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 336/670] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:55.652814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437374df-8309-42f0-aede-38902cd66e71 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.653026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437374df-8309-42f0-aede-38902cd66e71 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.653286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437374df-8309-42f0-aede-38902cd66e71 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.654499 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 337/671] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:55.655493 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bd8ac-73b0-4c6e-8a57-4fa1f2044821 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.655813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bd8ac-73b0-4c6e-8a57-4fa1f2044821 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.656301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491bd8ac-73b0-4c6e-8a57-4fa1f2044821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.675574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-491bd8ac-73b0-4c6e-8a57-4fa1f2044821 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:55.676784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-491bd8ac-73b0-4c6e-8a57-4fa1f2044821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:55.715992 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab81c1a-3f3b-476a-ae63-6ff927c1b0c3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:55.741951 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 338/672] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:55.742853 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bd8ac-73b0-4c6e-8a57-4fa1f2044821 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:55.762082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abb075-a982-43dc-b06b-e04ed8a0b356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:55.762689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abb075-a982-43dc-b06b-e04ed8a0b356 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.762968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abb075-a982-43dc-b06b-e04ed8a0b356 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.763238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6abb075-a982-43dc-b06b-e04ed8a0b356 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.764180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abb075-a982-43dc-b06b-e04ed8a0b356 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:55.764485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abb075-a982-43dc-b06b-e04ed8a0b356 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:55.765897 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6abb075-a982-43dc-b06b-e04ed8a0b356 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:55.768778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:55.778433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 335/673] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:55.779183 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abb075-a982-43dc-b06b-e04ed8a0b356 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:55.846470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:55.849960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491bd8ac-73b0-4c6e-8a57-4fa1f2044821 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:55.884739 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6abb075-a982-43dc-b06b-e04ed8a0b356 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:55.890433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8ac06e-485a-48db-a59d-921703ed5323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:55.891042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ac06e-485a-48db-a59d-921703ed5323 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.891325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ac06e-485a-48db-a59d-921703ed5323 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.891596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8ac06e-485a-48db-a59d-921703ed5323 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.893350 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ac06e-485a-48db-a59d-921703ed5323 None admin] RBAC: Adding query filter params (role.id=90322db354f94f03b32ec088922baf2e, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=e6dee005fc3d4fb593a1771e7851c20a) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:55.893662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ac06e-485a-48db-a59d-921703ed5323 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:55.895085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8ac06e-485a-48db-a59d-921703ed5323 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:55.911690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 339/674] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:55 2022] GET /identity/v3/role_assignments?user.id=e6dee005fc3d4fb593a1771e7851c20a&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=90322db354f94f03b32ec088922baf2e => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:55.911690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01a268-1e04-48c7-8e3b-d164a5d03884 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.911690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01a268-1e04-48c7-8e3b-d164a5d03884 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.911690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e01a268-1e04-48c7-8e3b-d164a5d03884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.925500 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1e01a268-1e04-48c7-8e3b-d164a5d03884 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:55.926728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1e01a268-1e04-48c7-8e3b-d164a5d03884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:55.937099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e21fd3-6f1b-48e1-b11e-1c4db3fafc02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:55.938256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e21fd3-6f1b-48e1-b11e-1c4db3fafc02 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:55.938540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e21fd3-6f1b-48e1-b11e-1c4db3fafc02 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:55.938843 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e21fd3-6f1b-48e1-b11e-1c4db3fafc02 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:55.940801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e21fd3-6f1b-48e1-b11e-1c4db3fafc02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:55.941532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e21fd3-6f1b-48e1-b11e-1c4db3fafc02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:55.943407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e21fd3-6f1b-48e1-b11e-1c4db3fafc02 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:55.964058 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 336/675] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:55.965005 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e21fd3-6f1b-48e1-b11e-1c4db3fafc02 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:56.023620 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 340/676] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:56.024586 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01a268-1e04-48c7-8e3b-d164a5d03884 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:56.047189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:56.106433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e21fd3-6f1b-48e1-b11e-1c4db3fafc02 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:56.147635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:56.149211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff34a0-d421-4fb6-b081-723882b59571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:56.150077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff34a0-d421-4fb6-b081-723882b59571 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:56.150580 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff34a0-d421-4fb6-b081-723882b59571 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:56.151046 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff34a0-d421-4fb6-b081-723882b59571 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:56.153939 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e01a268-1e04-48c7-8e3b-d164a5d03884 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:56.162141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff34a0-d421-4fb6-b081-723882b59571 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:56.163998 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff34a0-d421-4fb6-b081-723882b59571 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:56.176920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ceff34a0-d421-4fb6-b081-723882b59571 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:56.176920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:56.176920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:56.176920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:56.176920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:56.176920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:56.176920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:56.176920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:56.176920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:56.176920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:56.176920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:56.176920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:56.176920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:56.176920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:56.176920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:56.177835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:56.178731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:56.178731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:56.178731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:56.178731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:56.178731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:56.178731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:56.178731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:56.178731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:56.180698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 337/677] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:56.181621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff34a0-d421-4fb6-b081-723882b59571 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:56.206749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbfd40-bc26-4e53-9981-d72ce2739fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:56.207665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbfd40-bc26-4e53-9981-d72ce2739fec None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:56.208099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbfd40-bc26-4e53-9981-d72ce2739fec None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:56.208518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbbfd40-bc26-4e53-9981-d72ce2739fec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:56.209407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbfd40-bc26-4e53-9981-d72ce2739fec None admin] RBAC: Adding query filter params (role.id=90322db354f94f03b32ec088922baf2e, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=ef7e0b772a284e7da4fae7d73f34d3de) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:56.209719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbfd40-bc26-4e53-9981-d72ce2739fec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:56.214234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbbfd40-bc26-4e53-9981-d72ce2739fec None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:56.217600 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 341/678] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:56 2022] GET /identity/v3/role_assignments?user.id=ef7e0b772a284e7da4fae7d73f34d3de&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=90322db354f94f03b32ec088922baf2e => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:56.218501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbfd40-bc26-4e53-9981-d72ce2739fec None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:56.311191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff34a0-d421-4fb6-b081-723882b59571 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:56.332422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:56.352873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce470c28-37ed-43fc-9057-d57ba2d1442e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:56.353872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce470c28-37ed-43fc-9057-d57ba2d1442e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:56.354295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce470c28-37ed-43fc-9057-d57ba2d1442e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:56.354730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce470c28-37ed-43fc-9057-d57ba2d1442e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:56.364630 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbbfd40-bc26-4e53-9981-d72ce2739fec None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:56.372986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce470c28-37ed-43fc-9057-d57ba2d1442e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ce470c28-37ed-43fc-9057-d57ba2d1442e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:56.381171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:56.382184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:56.383001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:56.383001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:56.383001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:56.383001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:56.383001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:56.383001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:56.383001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:56.383001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:56.383001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:56.383001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:56.383001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:44:56.383001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:56.385173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 338/679] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:44:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:56.385875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce470c28-37ed-43fc-9057-d57ba2d1442e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:56.417763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5849ab7-bd46-4b80-8841-f0941b0515fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:56.418903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5849ab7-bd46-4b80-8841-f0941b0515fb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:56.419409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5849ab7-bd46-4b80-8841-f0941b0515fb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:56.419960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5849ab7-bd46-4b80-8841-f0941b0515fb None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:56.420878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5849ab7-bd46-4b80-8841-f0941b0515fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:56.421404 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5849ab7-bd46-4b80-8841-f0941b0515fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:56.423868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5849ab7-bd46-4b80-8841-f0941b0515fb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:56.437188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 342/680] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:56.438270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5849ab7-bd46-4b80-8841-f0941b0515fb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:56.464856 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:56.518495 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce470c28-37ed-43fc-9057-d57ba2d1442e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:56.573087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444bbac-6a29-4aab-8a1d-ea13031776b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:56.573916 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444bbac-6a29-4aab-8a1d-ea13031776b3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:56.574197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444bbac-6a29-4aab-8a1d-ea13031776b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:56.574471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d444bbac-6a29-4aab-8a1d-ea13031776b3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:56.575617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5849ab7-bd46-4b80-8841-f0941b0515fb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:56.593460 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d444bbac-6a29-4aab-8a1d-ea13031776b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d444bbac-6a29-4aab-8a1d-ea13031776b3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:44:56.598582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:56.599529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:56.600413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:56.600413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:56.600413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:56.600413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:56.600413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:44:56.600413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:44:56.600413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:44:56.600413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:44:56.600413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:44:56.600413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:44:56.600413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:44:56.600413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:56.600413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 339/681] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:44:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:56.601099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444bbac-6a29-4aab-8a1d-ea13031776b3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:56.625040 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e709b8-cc10-4a7e-91b4-58fae5d496c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:56.627251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e709b8-cc10-4a7e-91b4-58fae5d496c5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:56.627959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e709b8-cc10-4a7e-91b4-58fae5d496c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:56.628742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e709b8-cc10-4a7e-91b4-58fae5d496c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:56.630027 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e709b8-cc10-4a7e-91b4-58fae5d496c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:56.630769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e709b8-cc10-4a7e-91b4-58fae5d496c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:56.633058 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e709b8-cc10-4a7e-91b4-58fae5d496c5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:56.641042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:56.647442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 343/682] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:44:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:56.658606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e709b8-cc10-4a7e-91b4-58fae5d496c5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:56.719045 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d444bbac-6a29-4aab-8a1d-ea13031776b3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:56.755222 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec18555-c782-488c-8d6d-63226fafda2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:56.756033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec18555-c782-488c-8d6d-63226fafda2e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:56.756413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec18555-c782-488c-8d6d-63226fafda2e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:56.756782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec18555-c782-488c-8d6d-63226fafda2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:56.757616 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec18555-c782-488c-8d6d-63226fafda2e None admin] RBAC: Adding query filter params (role.id=90322db354f94f03b32ec088922baf2e, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=e6d98d51966f48d3a1b22f9b1c7963be) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:56.758024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec18555-c782-488c-8d6d-63226fafda2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:56.759559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec18555-c782-488c-8d6d-63226fafda2e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:56.765205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:56.766483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 340/683] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:56 2022] GET /identity/v3/role_assignments?user.id=e6d98d51966f48d3a1b22f9b1c7963be&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=90322db354f94f03b32ec088922baf2e => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:56.768082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec18555-c782-488c-8d6d-63226fafda2e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:56.791636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e709b8-cc10-4a7e-91b4-58fae5d496c5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:56.835057 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaa33c-719b-44ba-bc75-a6cd7d8fd4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:56.836198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaa33c-719b-44ba-bc75-a6cd7d8fd4e9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:56.837242 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaa33c-719b-44ba-bc75-a6cd7d8fd4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:56.837499 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecaa33c-719b-44ba-bc75-a6cd7d8fd4e9 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:56.838186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecaa33c-719b-44ba-bc75-a6cd7d8fd4e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:56.838487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecaa33c-719b-44ba-bc75-a6cd7d8fd4e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:56.840975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ecaa33c-719b-44ba-bc75-a6cd7d8fd4e9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:56.857865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 344/684] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:44:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:56.866339 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaa33c-719b-44ba-bc75-a6cd7d8fd4e9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:56.921353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:56.929053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec18555-c782-488c-8d6d-63226fafda2e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:56.995569 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c42ff8-afec-4383-ab70-2ebfb353692b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:56.996252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c42ff8-afec-4383-ab70-2ebfb353692b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:56.998794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c42ff8-afec-4383-ab70-2ebfb353692b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:56.999084 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c42ff8-afec-4383-ab70-2ebfb353692b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:57.026033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c42ff8-afec-4383-ab70-2ebfb353692b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:57.027545 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c42ff8-afec-4383-ab70-2ebfb353692b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-45c42ff8-afec-4383-ab70-2ebfb353692b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:57.055209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:57.056332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:57.057430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:57.057430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:57.057430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:57.059972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 341/685] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:57.065507 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c42ff8-afec-4383-ab70-2ebfb353692b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:57.085484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecaa33c-719b-44ba-bc75-a6cd7d8fd4e9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:57.140396 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56f764-a238-4525-9e02-4f05a12b9e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:57.141048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56f764-a238-4525-9e02-4f05a12b9e48 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:57.141330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56f764-a238-4525-9e02-4f05a12b9e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:57.141611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56f764-a238-4525-9e02-4f05a12b9e48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:57.155526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56f764-a238-4525-9e02-4f05a12b9e48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:57.158085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56f764-a238-4525-9e02-4f05a12b9e48 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:57.161048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5a56f764-a238-4525-9e02-4f05a12b9e48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:57.178198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:57.179328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:57.180262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:57.180262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:57.180262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:57.180801 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 345/686] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:57.186400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56f764-a238-4525-9e02-4f05a12b9e48 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:57.212939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c42ff8-afec-4383-ab70-2ebfb353692b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:57.258174 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21bb9c-890d-4ab4-a8da-11a59c40ab5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:57.259126 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21bb9c-890d-4ab4-a8da-11a59c40ab5b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:57.259587 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21bb9c-890d-4ab4-a8da-11a59c40ab5b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:57.260072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21bb9c-890d-4ab4-a8da-11a59c40ab5b None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:57.261000 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21bb9c-890d-4ab4-a8da-11a59c40ab5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:57.261481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21bb9c-890d-4ab4-a8da-11a59c40ab5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:57.263037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21bb9c-890d-4ab4-a8da-11a59c40ab5b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:57.289198 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 342/687] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:57.294498 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21bb9c-890d-4ab4-a8da-11a59c40ab5b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:57.317437 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56f764-a238-4525-9e02-4f05a12b9e48 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:57.322803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:57.360244 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18206c-4d01-4c0a-835c-66fd4dfefeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:57.361726 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18206c-4d01-4c0a-835c-66fd4dfefeba None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:57.362151 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18206c-4d01-4c0a-835c-66fd4dfefeba None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:57.362559 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f18206c-4d01-4c0a-835c-66fd4dfefeba None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:57.363332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18206c-4d01-4c0a-835c-66fd4dfefeba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:57.363808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18206c-4d01-4c0a-835c-66fd4dfefeba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:57.365380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f18206c-4d01-4c0a-835c-66fd4dfefeba None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:57.378854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 346/688] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:57.384065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18206c-4d01-4c0a-835c-66fd4dfefeba None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:57.471553 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21bb9c-890d-4ab4-a8da-11a59c40ab5b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:57.494597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:57.513178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a0349-c644-4255-944a-eed10074860a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:57.514420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a0349-c644-4255-944a-eed10074860a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:57.514894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a0349-c644-4255-944a-eed10074860a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:57.515522 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2a0349-c644-4255-944a-eed10074860a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:57.525454 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a0349-c644-4255-944a-eed10074860a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:57.527290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2a0349-c644-4255-944a-eed10074860a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-df2a0349-c644-4255-944a-eed10074860a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:57.541904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:57.544885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:57.545822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:57.545822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:57.545822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:57.545822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:57.545822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:57.545822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:57.545822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:57.545822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:57.545822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:44:57.545822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:57.545822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 343/689] 10.222.0.67 () {56 vars in 1093 bytes} [Mon Sep 26 17:44:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:44:57.553099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a0349-c644-4255-944a-eed10074860a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:57.599168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f18206c-4d01-4c0a-835c-66fd4dfefeba None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:57.670532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973aed1a-cdf0-4b68-93d4-4681a70054f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:57.671189 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973aed1a-cdf0-4b68-93d4-4681a70054f8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:57.671441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973aed1a-cdf0-4b68-93d4-4681a70054f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:57.674723 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973aed1a-cdf0-4b68-93d4-4681a70054f8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:57.680737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:57.693234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973aed1a-cdf0-4b68-93d4-4681a70054f8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:57.694773 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973aed1a-cdf0-4b68-93d4-4681a70054f8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-973aed1a-cdf0-4b68-93d4-4681a70054f8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:44:57.725749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:57.726927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:57.728091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:57.728091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:57.728091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:44:57.728091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:44:57.728091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:44:57.728091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:44:57.728091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:44:57.728091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:44:57.728091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:44:57.728091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:57.728091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 347/690] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:44:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:57.733504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973aed1a-cdf0-4b68-93d4-4681a70054f8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:57.735186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2a0349-c644-4255-944a-eed10074860a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:57.816615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3890a-35c3-41be-885a-4e775875a649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:57.817289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3890a-35c3-41be-885a-4e775875a649 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:57.817571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3890a-35c3-41be-885a-4e775875a649 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:57.817842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c3890a-35c3-41be-885a-4e775875a649 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:57.819578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3890a-35c3-41be-885a-4e775875a649 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:57.820093 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3890a-35c3-41be-885a-4e775875a649 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:57.821538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c3890a-35c3-41be-885a-4e775875a649 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:57.861695 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 344/691] 10.222.0.67 () {56 vars in 1133 bytes} [Mon Sep 26 17:44:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:57.868723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3890a-35c3-41be-885a-4e775875a649 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:57.906755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973aed1a-cdf0-4b68-93d4-4681a70054f8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:57.923005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:57.949317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e15743-dbc5-47c5-a4d6-55aea2c743e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:57.950333 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e15743-dbc5-47c5-a4d6-55aea2c743e3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:57.951385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e15743-dbc5-47c5-a4d6-55aea2c743e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:57.951912 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e15743-dbc5-47c5-a4d6-55aea2c743e3 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:57.954088 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e15743-dbc5-47c5-a4d6-55aea2c743e3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:57.955725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e15743-dbc5-47c5-a4d6-55aea2c743e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:57.957681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e15743-dbc5-47c5-a4d6-55aea2c743e3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:57.984312 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 348/692] 10.222.0.67 () {56 vars in 1135 bytes} [Mon Sep 26 17:44:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:44:57.992028 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c3890a-35c3-41be-885a-4e775875a649 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:57.996828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e15743-dbc5-47c5-a4d6-55aea2c743e3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:58.056447 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:58.069787 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ba0a3-4e6f-43d8-94bd-561bf47ee874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:58.071788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ba0a3-4e6f-43d8-94bd-561bf47ee874 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:58.071788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ba0a3-4e6f-43d8-94bd-561bf47ee874 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:58.071788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ba0a3-4e6f-43d8-94bd-561bf47ee874 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:58.173792 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e15743-dbc5-47c5-a4d6-55aea2c743e3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:58.185330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ba0a3-4e6f-43d8-94bd-561bf47ee874 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:58.185330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ba0a3-4e6f-43d8-94bd-561bf47ee874 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:58.253761 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c4cf2-01a2-4e2a-a6c8-edb85462f308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:58.254387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c4cf2-01a2-4e2a-a6c8-edb85462f308 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:58.254669 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c4cf2-01a2-4e2a-a6c8-edb85462f308 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:58.254944 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c4cf2-01a2-4e2a-a6c8-edb85462f308 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:58.324061 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 345/693] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:44:58.328528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ba0a3-4e6f-43d8-94bd-561bf47ee874 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:58.348009 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:58.386189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:58.390752 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c4cf2-01a2-4e2a-a6c8-edb85462f308 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:58.392826 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c4cf2-01a2-4e2a-a6c8-edb85462f308 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:58.510412 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ba0a3-4e6f-43d8-94bd-561bf47ee874 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:58.560157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059ac9d-106e-4411-b507-a429e3d9389f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:58.560797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059ac9d-106e-4411-b507-a429e3d9389f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:58.561077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059ac9d-106e-4411-b507-a429e3d9389f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:58.561350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059ac9d-106e-4411-b507-a429e3d9389f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:58.587239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059ac9d-106e-4411-b507-a429e3d9389f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:58.588852 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059ac9d-106e-4411-b507-a429e3d9389f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:58.601876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 349/694] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:44:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 26 17:44:58.603503 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05765995-16f6-4557-912f-2712674eddd2 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:58.604044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05765995-16f6-4557-912f-2712674eddd2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:58.604534 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05765995-16f6-4557-912f-2712674eddd2 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:58.605996 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 350/695] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:58.610705 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05765995-16f6-4557-912f-2712674eddd2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9059ac9d-106e-4411-b507-a429e3d9389f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:58.611418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:58.612375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:58.613250 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:58.613250 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:58.613250 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:58.613250 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 346/696] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:58.617279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf9347-979f-412d-b8f7-48becc3ce460 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:58.617868 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf9347-979f-412d-b8f7-48becc3ce460 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:58.618240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf9347-979f-412d-b8f7-48becc3ce460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:58.645002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-21cf9347-979f-412d-b8f7-48becc3ce460 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:58.647027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-21cf9347-979f-412d-b8f7-48becc3ce460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:58.706682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:58.734397 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 347/697] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:58.735397 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf9347-979f-412d-b8f7-48becc3ce460 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:58.746380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05765995-16f6-4557-912f-2712674eddd2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:58.761506 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:58.793487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2acd1-c5ca-47c7-a996-2099ff28de9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:58.794515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2acd1-c5ca-47c7-a996-2099ff28de9b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:58.794867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2acd1-c5ca-47c7-a996-2099ff28de9b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:58.795213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2acd1-c5ca-47c7-a996-2099ff28de9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:58.808791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2acd1-c5ca-47c7-a996-2099ff28de9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:58.810297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2acd1-c5ca-47c7-a996-2099ff28de9b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c5f2acd1-c5ca-47c7-a996-2099ff28de9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:58.825837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:58.826992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:58.828047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:58.828047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:58.828047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:58.829910 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 351/698] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:58.831322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae80a2-9c67-4934-87e3-e1001a5c9519 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:58.831775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae80a2-9c67-4934-87e3-e1001a5c9519 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:58.832158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae80a2-9c67-4934-87e3-e1001a5c9519 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:58.833789 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 352/699] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:58.840733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae80a2-9c67-4934-87e3-e1001a5c9519 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:58.859086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf9347-979f-412d-b8f7-48becc3ce460 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:58.905115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30097691-519b-468d-b1c9-8278c17ade05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:58.905848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30097691-519b-468d-b1c9-8278c17ade05 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:58.906128 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30097691-519b-468d-b1c9-8278c17ade05 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:58.906403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30097691-519b-468d-b1c9-8278c17ade05 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:58.907213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30097691-519b-468d-b1c9-8278c17ade05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:58.908156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30097691-519b-468d-b1c9-8278c17ade05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:58.910202 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30097691-519b-468d-b1c9-8278c17ade05 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:58.922201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 348/700] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:58.923702 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b56cf-7631-41a3-bad1-69a46058c0f5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:58.923990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b56cf-7631-41a3-bad1-69a46058c0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:58.924258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b56cf-7631-41a3-bad1-69a46058c0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:58.942008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-553b56cf-7631-41a3-bad1-69a46058c0f5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:44:58.943188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-553b56cf-7631-41a3-bad1-69a46058c0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:44:58.955048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae80a2-9c67-4934-87e3-e1001a5c9519 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:58.980980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:58.994991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da0899-6dc1-4d8c-ab90-3c1eaf62ad7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:58.996011 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da0899-6dc1-4d8c-ab90-3c1eaf62ad7b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.001907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da0899-6dc1-4d8c-ab90-3c1eaf62ad7b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.001907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da0899-6dc1-4d8c-ab90-3c1eaf62ad7b None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.001907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da0899-6dc1-4d8c-ab90-3c1eaf62ad7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:59.001907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da0899-6dc1-4d8c-ab90-3c1eaf62ad7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:59.001907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da0899-6dc1-4d8c-ab90-3c1eaf62ad7b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:59.019509 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 353/701] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:59.020892 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da0899-6dc1-4d8c-ab90-3c1eaf62ad7b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.024454 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 349/702] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:44:59.025323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b56cf-7631-41a3-bad1-69a46058c0f5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.027592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:59.131240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b56cf-7631-41a3-bad1-69a46058c0f5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.165252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da0899-6dc1-4d8c-ab90-3c1eaf62ad7b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.167262 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f886c4-c256-427b-9f2a-2711f16d9667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.168062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f886c4-c256-427b-9f2a-2711f16d9667 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.168336 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f886c4-c256-427b-9f2a-2711f16d9667 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.168601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f886c4-c256-427b-9f2a-2711f16d9667 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.176418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f886c4-c256-427b-9f2a-2711f16d9667 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:59.177863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f886c4-c256-427b-9f2a-2711f16d9667 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:59.189990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-43f886c4-c256-427b-9f2a-2711f16d9667 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:59.189990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:59.189990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:59.189990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:59.189990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:59.189990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:59.189990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:59.189990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:59.189990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:59.189990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:59.189990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:59.189990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:59.189990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:59.189990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:59.189990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:59.191073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:59.191961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:59.191961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:59.191961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:59.191961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:59.191961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:59.191961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:59.191961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:59.191961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:59.191961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 350/703] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:59.192431 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f886c4-c256-427b-9f2a-2711f16d9667 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.213450 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15382f-6a87-42a3-b03e-ab9f70dded25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.214482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15382f-6a87-42a3-b03e-ab9f70dded25 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.214960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15382f-6a87-42a3-b03e-ab9f70dded25 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.215514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e15382f-6a87-42a3-b03e-ab9f70dded25 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.226214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15382f-6a87-42a3-b03e-ab9f70dded25 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:59.228703 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e15382f-6a87-42a3-b03e-ab9f70dded25 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:59.247440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4e15382f-6a87-42a3-b03e-ab9f70dded25 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:59.247440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:59.247440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:59.247440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:59.247440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:59.247440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:59.247440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:59.247440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:59.247440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:59.247440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:59.247440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:59.247440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:59.247440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:44:59.247440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:44:59.247440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:59.248585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:59.249501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:44:59.249501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:44:59.249501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:44:59.249501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:44:59.249501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:44:59.249501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:44:59.249501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:44:59.249501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:59.252801 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 354/704] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:59.254067 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15382f-6a87-42a3-b03e-ab9f70dded25 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.260613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:59.304998 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f886c4-c256-427b-9f2a-2711f16d9667 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.320114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:59.346417 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9af99-1d80-432b-ac04-e61714633602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.347066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9af99-1d80-432b-ac04-e61714633602 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.347348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9af99-1d80-432b-ac04-e61714633602 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.347624 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9af99-1d80-432b-ac04-e61714633602 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.358466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9af99-1d80-432b-ac04-e61714633602 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:59.360245 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a9af99-1d80-432b-ac04-e61714633602 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d6a9af99-1d80-432b-ac04-e61714633602 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:44:59.376905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:59.378001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:44:59.378887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:44:59.378887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:44:59.378887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:44:59.378887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 351/705] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:59.379109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9af99-1d80-432b-ac04-e61714633602 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.396137 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e15382f-6a87-42a3-b03e-ab9f70dded25 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.436602 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40101c2-bc12-4ed2-af66-3bbdc8e4f001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.437219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40101c2-bc12-4ed2-af66-3bbdc8e4f001 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.437609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40101c2-bc12-4ed2-af66-3bbdc8e4f001 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.437890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40101c2-bc12-4ed2-af66-3bbdc8e4f001 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.440093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40101c2-bc12-4ed2-af66-3bbdc8e4f001 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:59.440743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40101c2-bc12-4ed2-af66-3bbdc8e4f001 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:59.442315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40101c2-bc12-4ed2-af66-3bbdc8e4f001 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:59.460001 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 355/706] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:59.460958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40101c2-bc12-4ed2-af66-3bbdc8e4f001 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.493818 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9af99-1d80-432b-ac04-e61714633602 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.509782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:59.530656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126447a-b876-4b7e-a480-2e9cbcac921a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.531321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126447a-b876-4b7e-a480-2e9cbcac921a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.531603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126447a-b876-4b7e-a480-2e9cbcac921a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.532452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126447a-b876-4b7e-a480-2e9cbcac921a None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.532682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126447a-b876-4b7e-a480-2e9cbcac921a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:59.533193 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126447a-b876-4b7e-a480-2e9cbcac921a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:59.534764 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126447a-b876-4b7e-a480-2e9cbcac921a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:59.552253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 352/707] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:59.553170 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126447a-b876-4b7e-a480-2e9cbcac921a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.583263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40101c2-bc12-4ed2-af66-3bbdc8e4f001 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.586445 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:59.620205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9bae1-add7-458d-9d91-5fb2d5245ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.620833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9bae1-add7-458d-9d91-5fb2d5245ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.621109 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9bae1-add7-458d-9d91-5fb2d5245ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.621373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b9bae1-add7-458d-9d91-5fb2d5245ae1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.621944 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b9bae1-add7-458d-9d91-5fb2d5245ae1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:59.623286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b9bae1-add7-458d-9d91-5fb2d5245ae1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a6b9bae1-add7-458d-9d91-5fb2d5245ae1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:44:59.634742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:44:59.635793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:44:59.636611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:44:59.636611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:44:59.636611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:44:59.636611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:44:59.636611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:44:59.636611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:44:59.636611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:44:59.636611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:44:59.636611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:44:59.636611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:44:59.636611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:44:59.636611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:44:59.636611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:44:59.636611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:44:59.636611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 356/708] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/services/network => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:44:59.638037 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9bae1-add7-458d-9d91-5fb2d5245ae1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.656088 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126447a-b876-4b7e-a480-2e9cbcac921a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.691083 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9dd163-d84d-4ea5-8a9a-a07d341b275a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.691714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9dd163-d84d-4ea5-8a9a-a07d341b275a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.691995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9dd163-d84d-4ea5-8a9a-a07d341b275a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.692266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9dd163-d84d-4ea5-8a9a-a07d341b275a None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.692889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9dd163-d84d-4ea5-8a9a-a07d341b275a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:59.693185 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9dd163-d84d-4ea5-8a9a-a07d341b275a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:59.694619 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9dd163-d84d-4ea5-8a9a-a07d341b275a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:59.706395 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 353/709] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:59.707460 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9dd163-d84d-4ea5-8a9a-a07d341b275a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.741669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:59.742685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b9bae1-add7-458d-9d91-5fb2d5245ae1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.777235 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94a968-5fd7-476a-8fcc-069156a98b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.777829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94a968-5fd7-476a-8fcc-069156a98b72 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.778105 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94a968-5fd7-476a-8fcc-069156a98b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.778378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94a968-5fd7-476a-8fcc-069156a98b72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.779083 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94a968-5fd7-476a-8fcc-069156a98b72 None admin] RBAC: Adding query filter params (role.id=90322db354f94f03b32ec088922baf2e, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=7ba12b94717a4a40b55635192840d3c3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:59.779389 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94a968-5fd7-476a-8fcc-069156a98b72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:59.780829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94a968-5fd7-476a-8fcc-069156a98b72 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:59.787332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 357/710] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/role_assignments?user.id=7ba12b94717a4a40b55635192840d3c3&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=90322db354f94f03b32ec088922baf2e => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:59.788326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e66700-49c3-4806-9e89-e5b4d4b7e1a3 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.788606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e66700-49c3-4806-9e89-e5b4d4b7e1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.788877 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e66700-49c3-4806-9e89-e5b4d4b7e1a3 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.790053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 358/711] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:44:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:44:59.790736 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e66700-49c3-4806-9e89-e5b4d4b7e1a3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.827817 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9dd163-d84d-4ea5-8a9a-a07d341b275a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.828461 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:44:59.874276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069226e0-dae8-4096-a464-b46719a5f049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.874900 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069226e0-dae8-4096-a464-b46719a5f049 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.875174 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069226e0-dae8-4096-a464-b46719a5f049 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.875440 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069226e0-dae8-4096-a464-b46719a5f049 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.876747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069226e0-dae8-4096-a464-b46719a5f049 None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=b9ed29a3f6be4d64bb3751f9e95eb56b) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:59.877056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069226e0-dae8-4096-a464-b46719a5f049 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:59.878554 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069226e0-dae8-4096-a464-b46719a5f049 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:59.885472 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 354/712] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/role_assignments?user.id=b9ed29a3f6be4d64bb3751f9e95eb56b&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=07626899472b4e32adc0c71ce796a25d => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:59.886316 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069226e0-dae8-4096-a464-b46719a5f049 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:44:59.929255 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e66700-49c3-4806-9e89-e5b4d4b7e1a3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:44:59.984704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c897949-dc73-472a-aca1-317a0f3605ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:44:59.988704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c897949-dc73-472a-aca1-317a0f3605ac None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.988704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c897949-dc73-472a-aca1-317a0f3605ac None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:44:59.988704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c897949-dc73-472a-aca1-317a0f3605ac None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:44:59.988704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c897949-dc73-472a-aca1-317a0f3605ac None admin] RBAC: Adding query filter params (name=network) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:44:59.988704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c897949-dc73-472a-aca1-317a0f3605ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:44:59.989278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c897949-dc73-472a-aca1-317a0f3605ac None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:44:59.996579 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 359/713] 10.222.0.67 () {56 vars in 1112 bytes} [Mon Sep 26 17:44:59 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:44:59.998614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ad643-0445-4b6e-9214-42d2088f3475 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:44:59.999777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ad643-0445-4b6e-9214-42d2088f3475 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:00.000312 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ad643-0445-4b6e-9214-42d2088f3475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:00.025387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2c3ad643-0445-4b6e-9214-42d2088f3475 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:00.027338 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2c3ad643-0445-4b6e-9214-42d2088f3475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:00.043008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:00.047766 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069226e0-dae8-4096-a464-b46719a5f049 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:00.089361 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41f75c-e65d-422e-8bc4-1d3f2cabf27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:00.089973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41f75c-e65d-422e-8bc4-1d3f2cabf27e None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:00.090252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41f75c-e65d-422e-8bc4-1d3f2cabf27e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:00.090530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41f75c-e65d-422e-8bc4-1d3f2cabf27e None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:00.091175 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41f75c-e65d-422e-8bc4-1d3f2cabf27e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:00.092657 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41f75c-e65d-422e-8bc4-1d3f2cabf27e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:00.119289 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 360/714] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:44:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:00.120555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ad643-0445-4b6e-9214-42d2088f3475 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:00.150394 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:00.192017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 355/715] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:44:59 2022] POST /identity/v3/users => generated 254 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:00.246246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ad643-0445-4b6e-9214-42d2088f3475 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:00.300422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c527c-6c86-475a-996f-96147bdcab97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:00.301339 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c527c-6c86-475a-996f-96147bdcab97 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:00.301771 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c527c-6c86-475a-996f-96147bdcab97 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:00.302193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131c527c-6c86-475a-996f-96147bdcab97 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:00.302978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c527c-6c86-475a-996f-96147bdcab97 None admin] RBAC: Adding query filter params (type=network) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:00.303463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c527c-6c86-475a-996f-96147bdcab97 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:00.305040 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131c527c-6c86-475a-996f-96147bdcab97 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:00.315930 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 361/716] 10.222.0.67 () {56 vars in 1112 bytes} [Mon Sep 26 17:45:00 2022] GET /identity/v3/services?type=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:00.474734 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41f75c-e65d-422e-8bc4-1d3f2cabf27e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:00.590826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41f75c-e65d-422e-8bc4-1d3f2cabf27e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:00.629337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8311d-0e60-4719-87fd-c5ab4f1ab202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:00.630231 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8311d-0e60-4719-87fd-c5ab4f1ab202 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:00.630665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8311d-0e60-4719-87fd-c5ab4f1ab202 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:00.631118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b8311d-0e60-4719-87fd-c5ab4f1ab202 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:00.631889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8311d-0e60-4719-87fd-c5ab4f1ab202 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:00.633438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b8311d-0e60-4719-87fd-c5ab4f1ab202 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e0b8311d-0e60-4719-87fd-c5ab4f1ab202 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:00.647041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:00.648294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:00.649362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:00.649362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:00.649362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:00.649362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:00.649362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:00.649362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:00.649362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:00.649362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:00.649362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:00.649362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:00.649362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:00.649362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:00.649362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:45:00.649362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:00.652001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 356/717] 10.222.0.67 () {56 vars in 1129 bytes} [Mon Sep 26 17:45:00 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:00.657729 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c527c-6c86-475a-996f-96147bdcab97 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:00.760299 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:00.772127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131c527c-6c86-475a-996f-96147bdcab97 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:00.817115 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8e60a-6fa5-4e17-85eb-260699704ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:00.818076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8e60a-6fa5-4e17-85eb-260699704ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:00.818508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8e60a-6fa5-4e17-85eb-260699704ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:00.818931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8e60a-6fa5-4e17-85eb-260699704ba7 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:00.819727 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8e60a-6fa5-4e17-85eb-260699704ba7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:00.820179 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8e60a-6fa5-4e17-85eb-260699704ba7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:00.821720 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8e60a-6fa5-4e17-85eb-260699704ba7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:00.831874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 362/718] 10.222.0.67 () {56 vars in 1124 bytes} [Mon Sep 26 17:45:00 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:00.837666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8311d-0e60-4719-87fd-c5ab4f1ab202 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:00.863380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:00.969343 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b8311d-0e60-4719-87fd-c5ab4f1ab202 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:01.022650 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a184dac-a473-4c78-b9b3-ced4d82e8837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:01.023498 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a184dac-a473-4c78-b9b3-ced4d82e8837 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:01.023942 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a184dac-a473-4c78-b9b3-ced4d82e8837 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:01.024354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a184dac-a473-4c78-b9b3-ced4d82e8837 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:01.025141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a184dac-a473-4c78-b9b3-ced4d82e8837 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:01.025583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a184dac-a473-4c78-b9b3-ced4d82e8837 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:01.027101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a184dac-a473-4c78-b9b3-ced4d82e8837 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:01.033675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 357/719] 10.222.0.67 () {56 vars in 1124 bytes} [Mon Sep 26 17:45:00 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:02.374073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866bd65-54ea-4030-a74e-60803df7ddab None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.374810 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866bd65-54ea-4030-a74e-60803df7ddab None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.374810 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866bd65-54ea-4030-a74e-60803df7ddab None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:02.376778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 363/720] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:02.382793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a4fa9-5cef-41b2-9574-8e63f912fcad None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.383988 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a4fa9-5cef-41b2-9574-8e63f912fcad None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.384699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a4fa9-5cef-41b2-9574-8e63f912fcad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:02.417520 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c70a4fa9-5cef-41b2-9574-8e63f912fcad None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:02.419740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c70a4fa9-5cef-41b2-9574-8e63f912fcad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:02.508634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 358/721] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:02.840872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866bd65-54ea-4030-a74e-60803df7ddab None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:02.971412 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c44fe-0835-43c3-9f59-a0094f80ee88 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.972144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c44fe-0835-43c3-9f59-a0094f80ee88 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.972635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c44fe-0835-43c3-9f59-a0094f80ee88 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:02.975233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 359/722] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:02.983143 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dd0e4-712b-4dfd-9670-e5c34ca026ec None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:02.983719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dd0e4-712b-4dfd-9670-e5c34ca026ec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:02.984204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dd0e4-712b-4dfd-9670-e5c34ca026ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.005708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866bd65-54ea-4030-a74e-60803df7ddab None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:03.012579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-931dd0e4-712b-4dfd-9670-e5c34ca026ec None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:03.017699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-931dd0e4-712b-4dfd-9670-e5c34ca026ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:03.037255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:03.056437 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cf049-c7a7-4ae8-ad04-202bda316f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:03.057507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cf049-c7a7-4ae8-ad04-202bda316f21 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.058147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cf049-c7a7-4ae8-ad04-202bda316f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.058631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cf049-c7a7-4ae8-ad04-202bda316f21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.070595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519cf049-c7a7-4ae8-ad04-202bda316f21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-519cf049-c7a7-4ae8-ad04-202bda316f21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:03.076654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:03.077610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:03.078487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:03.078487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:03.078487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:03.078487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:03.078487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:03.078487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:03.078487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:03.078487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:03.078487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:03.078487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:03.078487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:03.078487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:03.081322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 364/723] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:03.082734 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0671f7-4bf9-4657-a6e9-c3f57a98c011 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.083246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0671f7-4bf9-4657-a6e9-c3f57a98c011 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.083826 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0671f7-4bf9-4657-a6e9-c3f57a98c011 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.085530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 365/724] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:03.086512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0671f7-4bf9-4657-a6e9-c3f57a98c011 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:03.099529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 360/725] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 26 17:45:03.103685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dede9b-8b18-4732-8211-425b3ee16142 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.103685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dede9b-8b18-4732-8211-425b3ee16142 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.103685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dede9b-8b18-4732-8211-425b3ee16142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.115679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:03.132402 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-34dede9b-8b18-4732-8211-425b3ee16142 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:03.134832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-34dede9b-8b18-4732-8211-425b3ee16142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:03.281338 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 361/726] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:03.289403 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0671f7-4bf9-4657-a6e9-c3f57a98c011 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:03.300176 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c36205-7b6b-4111-b1a8-4f47758d7c3a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.300458 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c36205-7b6b-4111-b1a8-4f47758d7c3a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.300731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c36205-7b6b-4111-b1a8-4f47758d7c3a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.302058 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 362/727] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:03.307293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd893b28-91e1-44df-aaa2-734b41a80f70 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.307567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd893b28-91e1-44df-aaa2-734b41a80f70 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.307878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd893b28-91e1-44df-aaa2-734b41a80f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.339187 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-dd893b28-91e1-44df-aaa2-734b41a80f70 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:03.340395 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dd893b28-91e1-44df-aaa2-734b41a80f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:03.368515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d8dff-9cad-4326-9e82-125691bbb7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:03.369131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d8dff-9cad-4326-9e82-125691bbb7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.369410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d8dff-9cad-4326-9e82-125691bbb7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.369679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d8dff-9cad-4326-9e82-125691bbb7a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.370339 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d8dff-9cad-4326-9e82-125691bbb7a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:03.370637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d8dff-9cad-4326-9e82-125691bbb7a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:03.372023 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82d8dff-9cad-4326-9e82-125691bbb7a7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:03.394554 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 366/728] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:03.400035 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d8dff-9cad-4326-9e82-125691bbb7a7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:03.414784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 363/729] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 26 17:45:03.440337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd893b28-91e1-44df-aaa2-734b41a80f70 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:03.442903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:03.520563 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:03.562913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d8dff-9cad-4326-9e82-125691bbb7a7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:03.607373 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd893b28-91e1-44df-aaa2-734b41a80f70 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:03.631593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8f107-c923-4e5d-ad61-8fcf8ddc463f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:03.632438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8f107-c923-4e5d-ad61-8fcf8ddc463f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.632724 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8f107-c923-4e5d-ad61-8fcf8ddc463f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.632991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb8f107-c923-4e5d-ad61-8fcf8ddc463f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.646037 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8f107-c923-4e5d-ad61-8fcf8ddc463f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:03.647566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb8f107-c923-4e5d-ad61-8fcf8ddc463f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:03.662018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5324259-9e34-4b9e-a2b4-8814b5d11110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:03.662665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5324259-9e34-4b9e-a2b4-8814b5d11110 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.662945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5324259-9e34-4b9e-a2b4-8814b5d11110 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.663219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5324259-9e34-4b9e-a2b4-8814b5d11110 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ddb8f107-c923-4e5d-ad61-8fcf8ddc463f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:03.663945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:03.664835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:03.665706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:03.665706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:03.665706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:03.665706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 367/730] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:03.666383 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8f107-c923-4e5d-ad61-8fcf8ddc463f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:03.680263 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5324259-9e34-4b9e-a2b4-8814b5d11110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c5324259-9e34-4b9e-a2b4-8814b5d11110 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:03.685604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:03.686538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:03.687345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:03.687345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:03.687345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:03.687345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:03.687345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:03.687345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:03.687345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:03.687345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:03.687345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:03.687345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:03.687345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:03.687345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:03.687345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 364/731] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:03.692133 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5324259-9e34-4b9e-a2b4-8814b5d11110 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:03.758243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:03.786599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb8f107-c923-4e5d-ad61-8fcf8ddc463f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:03.798598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5324259-9e34-4b9e-a2b4-8814b5d11110 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:03.819492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:03.827133 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc4c22-7772-447b-82ee-842576652bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:03.827770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc4c22-7772-447b-82ee-842576652bcc None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.828048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc4c22-7772-447b-82ee-842576652bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.828315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bc4c22-7772-447b-82ee-842576652bcc None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.829177 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc4c22-7772-447b-82ee-842576652bcc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:03.831219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bc4c22-7772-447b-82ee-842576652bcc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:03.833899 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e5541f-619d-4d74-8a32-22d70f6209a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:03.834517 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5541f-619d-4d74-8a32-22d70f6209a9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:03.834796 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5541f-619d-4d74-8a32-22d70f6209a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:03.835068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5541f-619d-4d74-8a32-22d70f6209a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:03.835764 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e5541f-619d-4d74-8a32-22d70f6209a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:03.836076 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e5541f-619d-4d74-8a32-22d70f6209a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:03.837521 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e5541f-619d-4d74-8a32-22d70f6209a9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:03.849353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 365/732] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:03.850031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e5541f-619d-4d74-8a32-22d70f6209a9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:03.943086 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 368/733] 10.222.0.67 () {60 vars in 1141 bytes} [Mon Sep 26 17:45:03 2022] POST /identity/v3/services => generated 240 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:03.943981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc4c22-7772-447b-82ee-842576652bcc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:03.953970 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e5541f-619d-4d74-8a32-22d70f6209a9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:03.980852 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:04.004017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbfca39-392c-4689-8371-1d224d775f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:04.004986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfca39-392c-4689-8371-1d224d775f59 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.005336 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfca39-392c-4689-8371-1d224d775f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.005622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbfca39-392c-4689-8371-1d224d775f59 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.006314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfca39-392c-4689-8371-1d224d775f59 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:04.006621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfca39-392c-4689-8371-1d224d775f59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:04.008127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbfca39-392c-4689-8371-1d224d775f59 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:04.019874 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 366/734] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:04.021032 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce513a8c-701b-499e-917a-489fbf8d0830 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.021375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce513a8c-701b-499e-917a-489fbf8d0830 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.021651 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce513a8c-701b-499e-917a-489fbf8d0830 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.027433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 367/735] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:04.027433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce513a8c-701b-499e-917a-489fbf8d0830 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:04.080088 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bc4c22-7772-447b-82ee-842576652bcc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:04.123503 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de524aa4-d7af-4477-a143-05171d21f178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:04.124665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de524aa4-d7af-4477-a143-05171d21f178 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.125139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de524aa4-d7af-4477-a143-05171d21f178 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.125612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de524aa4-d7af-4477-a143-05171d21f178 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.139728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de524aa4-d7af-4477-a143-05171d21f178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-de524aa4-d7af-4477-a143-05171d21f178 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:04.145246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:04.146156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:04.147015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:04.147015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:04.147015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:04.147015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:04.147015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:04.147015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:04.147015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:04.147015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:04.147015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:04.147015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:04.147015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:04.147015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:04.147015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 369/736] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:45:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:04.148344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de524aa4-d7af-4477-a143-05171d21f178 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:04.154800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce513a8c-701b-499e-917a-489fbf8d0830 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:04.192124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a97ea-302c-4ae9-b63f-7c01069e1cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:04.192760 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a97ea-302c-4ae9-b63f-7c01069e1cac None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.193023 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a97ea-302c-4ae9-b63f-7c01069e1cac None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.193344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a97ea-302c-4ae9-b63f-7c01069e1cac None admin] PATH_INFO: `/v3/users/swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.200776 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a97ea-302c-4ae9-b63f-7c01069e1cac None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:04.202313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257a97ea-302c-4ae9-b63f-7c01069e1cac None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:04.202700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-257a97ea-302c-4ae9-b63f-7c01069e1cac None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:04.214841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:04.215907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:04.216826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:04.216826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:04.216826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:04.216826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:04.216826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:04.216826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:04.216826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:04.216826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:04.216826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:45:04.216826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:04.216826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 368/737] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:04.217391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41ee74-f3dd-49c7-84cc-16336fe0602a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.217391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41ee74-f3dd-49c7-84cc-16336fe0602a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.218017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41ee74-f3dd-49c7-84cc-16336fe0602a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.237640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2b41ee74-f3dd-49c7-84cc-16336fe0602a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:04.238896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2b41ee74-f3dd-49c7-84cc-16336fe0602a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:04.261264 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de524aa4-d7af-4477-a143-05171d21f178 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:04.274745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:04.299003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d047c83-4bc0-4954-b68d-55732ec5e1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:04.299620 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d047c83-4bc0-4954-b68d-55732ec5e1ed None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.299922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d047c83-4bc0-4954-b68d-55732ec5e1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.300239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d047c83-4bc0-4954-b68d-55732ec5e1ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.315690 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 369/738] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:04 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 26 17:45:04.315873 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d047c83-4bc0-4954-b68d-55732ec5e1ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:04.315967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41ee74-f3dd-49c7-84cc-16336fe0602a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:04.316051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d047c83-4bc0-4954-b68d-55732ec5e1ed None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0d047c83-4bc0-4954-b68d-55732ec5e1ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:04.336802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:04.337949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:04.338847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:04.338847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:04.338847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:04.338847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 370/739] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:04.339592 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d047c83-4bc0-4954-b68d-55732ec5e1ed None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:04.448396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41ee74-f3dd-49c7-84cc-16336fe0602a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:04.475519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:04.480079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d047c83-4bc0-4954-b68d-55732ec5e1ed None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:04.485370 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7fd4b-cf65-478a-9c02-c3f76057ac1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:04.486186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7fd4b-cf65-478a-9c02-c3f76057ac1d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.490703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7fd4b-cf65-478a-9c02-c3f76057ac1d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.490703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef7fd4b-cf65-478a-9c02-c3f76057ac1d None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.490703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7fd4b-cf65-478a-9c02-c3f76057ac1d None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:04.490703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7fd4b-cf65-478a-9c02-c3f76057ac1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:04.490703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef7fd4b-cf65-478a-9c02-c3f76057ac1d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:04.498919 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 370/740] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:45:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:04.500124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7fd4b-cf65-478a-9c02-c3f76057ac1d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:04.522572 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac47208-0e55-4a0a-abbe-e151572bada9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:04.523188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac47208-0e55-4a0a-abbe-e151572bada9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.523513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac47208-0e55-4a0a-abbe-e151572bada9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.523842 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac47208-0e55-4a0a-abbe-e151572bada9 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.524563 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac47208-0e55-4a0a-abbe-e151572bada9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:04.524866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac47208-0e55-4a0a-abbe-e151572bada9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:04.526823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac47208-0e55-4a0a-abbe-e151572bada9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:04.537374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:04.548628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 371/741] 10.222.0.67 () {56 vars in 1135 bytes} [Mon Sep 26 17:45:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:04.549675 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac47208-0e55-4a0a-abbe-e151572bada9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:04.631463 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef7fd4b-cf65-478a-9c02-c3f76057ac1d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:04.659154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac47208-0e55-4a0a-abbe-e151572bada9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:04.668594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b84ce6-e9af-4cd7-867a-81055ea681dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:04.669201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b84ce6-e9af-4cd7-867a-81055ea681dd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.669476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b84ce6-e9af-4cd7-867a-81055ea681dd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.669744 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b84ce6-e9af-4cd7-867a-81055ea681dd None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.670501 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b84ce6-e9af-4cd7-867a-81055ea681dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:04.670797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b84ce6-e9af-4cd7-867a-81055ea681dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:04.672256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b84ce6-e9af-4cd7-867a-81055ea681dd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:04.691282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 371/742] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:04.692016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b84ce6-e9af-4cd7-867a-81055ea681dd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:04.696657 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cd08f-9fbf-4a18-9e14-71e612402d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:04.697305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cd08f-9fbf-4a18-9e14-71e612402d18 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.697584 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cd08f-9fbf-4a18-9e14-71e612402d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.697855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49cd08f-9fbf-4a18-9e14-71e612402d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.709519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cd08f-9fbf-4a18-9e14-71e612402d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:04.710890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49cd08f-9fbf-4a18-9e14-71e612402d18 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a49cd08f-9fbf-4a18-9e14-71e612402d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:04.726948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:04.727964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:04.728827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:04.728827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:04.728827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:04.728827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 372/743] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:04.728827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cd08f-9fbf-4a18-9e14-71e612402d18 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:04.743257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:04.798354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:04.808229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b84ce6-e9af-4cd7-867a-81055ea681dd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:04.837836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49cd08f-9fbf-4a18-9e14-71e612402d18 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:04.847110 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff0f97-e6bb-4905-a7c7-fa12faaf1c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:04.847760 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff0f97-e6bb-4905-a7c7-fa12faaf1c35 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.848044 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff0f97-e6bb-4905-a7c7-fa12faaf1c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.848316 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff0f97-e6bb-4905-a7c7-fa12faaf1c35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.879970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b6c9e-26d7-434f-9d98-73e82be27248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:04.880596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b6c9e-26d7-434f-9d98-73e82be27248 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:04.880890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b6c9e-26d7-434f-9d98-73e82be27248 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:04.881150 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b6c9e-26d7-434f-9d98-73e82be27248 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:04.881808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b6c9e-26d7-434f-9d98-73e82be27248 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:04.883185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b6c9e-26d7-434f-9d98-73e82be27248 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:04.919056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff0f97-e6bb-4905-a7c7-fa12faaf1c35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:04.920603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff0f97-e6bb-4905-a7c7-fa12faaf1c35 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:04.944418 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 373/744] 10.222.0.67 () {60 vars in 1141 bytes} [Mon Sep 26 17:45:04 2022] POST /identity/v3/services => generated 251 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:04.945443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b6c9e-26d7-434f-9d98-73e82be27248 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:05.052393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:05.059374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:05.073555 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 372/745] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:04 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 26 17:45:05.074738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff0f97-e6bb-4905-a7c7-fa12faaf1c35 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:05.106830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b6c9e-26d7-434f-9d98-73e82be27248 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:05.155342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9598bd6-c041-45db-8028-e8aae49c8ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:05.156026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9598bd6-c041-45db-8028-e8aae49c8ab9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:05.156307 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9598bd6-c041-45db-8028-e8aae49c8ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:05.156580 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9598bd6-c041-45db-8028-e8aae49c8ab9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:05.169536 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9598bd6-c041-45db-8028-e8aae49c8ab9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:05.170846 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9598bd6-c041-45db-8028-e8aae49c8ab9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a9598bd6-c041-45db-8028-e8aae49c8ab9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:05.184520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:05.185504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:05.186430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:05.186430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:05.186430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:05.186430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:05.186430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:05.186430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:05.186430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:05.186430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:05.186430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:45:05.186430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:05.186430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 374/746] 10.222.0.67 () {56 vars in 1093 bytes} [Mon Sep 26 17:45:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:05.187028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9598bd6-c041-45db-8028-e8aae49c8ab9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:05.208077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff0f97-e6bb-4905-a7c7-fa12faaf1c35 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:05.255666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b478d-94b2-4adb-9a28-9f9210341eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:05.256831 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b478d-94b2-4adb-9a28-9f9210341eb4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:05.257314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b478d-94b2-4adb-9a28-9f9210341eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:05.258009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936b478d-94b2-4adb-9a28-9f9210341eb4 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:05.259254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b478d-94b2-4adb-9a28-9f9210341eb4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:05.260005 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b478d-94b2-4adb-9a28-9f9210341eb4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:05.262213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936b478d-94b2-4adb-9a28-9f9210341eb4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:05.283165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 373/747] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:05.288530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b478d-94b2-4adb-9a28-9f9210341eb4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:05.323294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9598bd6-c041-45db-8028-e8aae49c8ab9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:05.349860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:05.359788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:05.372156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efdc66-fa54-4524-ab31-225d3cbf4335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:05.373123 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efdc66-fa54-4524-ab31-225d3cbf4335 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:05.373595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efdc66-fa54-4524-ab31-225d3cbf4335 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:05.374047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efdc66-fa54-4524-ab31-225d3cbf4335 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:05.387755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64efdc66-fa54-4524-ab31-225d3cbf4335 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:05.389989 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64efdc66-fa54-4524-ab31-225d3cbf4335 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:05.403564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936b478d-94b2-4adb-9a28-9f9210341eb4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-64efdc66-fa54-4524-ab31-225d3cbf4335 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:05.410419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:05.411566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:05.412698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:05.412698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:05.412698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:05.415128 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 375/748] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:05.416277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efdc66-fa54-4524-ab31-225d3cbf4335 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:05.445362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f1a85-b66d-461b-ba0c-a0d6ac34fe43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:05.445794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f1a85-b66d-461b-ba0c-a0d6ac34fe43 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:05.446155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f1a85-b66d-461b-ba0c-a0d6ac34fe43 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:05.446553 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f1a85-b66d-461b-ba0c-a0d6ac34fe43 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:05.447358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f1a85-b66d-461b-ba0c-a0d6ac34fe43 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:05.447813 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f1a85-b66d-461b-ba0c-a0d6ac34fe43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:05.449323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f1a85-b66d-461b-ba0c-a0d6ac34fe43 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:05.475016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 374/749] 10.222.0.67 () {56 vars in 1133 bytes} [Mon Sep 26 17:45:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:05.475016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f1a85-b66d-461b-ba0c-a0d6ac34fe43 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:05.551901 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efdc66-fa54-4524-ab31-225d3cbf4335 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:05.589480 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40c9ae-3888-419f-a0e5-a4e437e4f60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:05.590274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40c9ae-3888-419f-a0e5-a4e437e4f60b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:05.590692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40c9ae-3888-419f-a0e5-a4e437e4f60b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:05.591077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c40c9ae-3888-419f-a0e5-a4e437e4f60b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:05.601172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40c9ae-3888-419f-a0e5-a4e437e4f60b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:05.602757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c40c9ae-3888-419f-a0e5-a4e437e4f60b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1c40c9ae-3888-419f-a0e5-a4e437e4f60b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:05.615813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:05.616959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:05.618118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:05.618118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:05.618118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:05.618118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:05.618118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:05.618118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:05.618118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:05.618118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:05.618118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:45:05.618118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:05.622327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f1a85-b66d-461b-ba0c-a0d6ac34fe43 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:05.624296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 376/750] 10.222.0.67 () {56 vars in 1117 bytes} [Mon Sep 26 17:45:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:05.625733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40c9ae-3888-419f-a0e5-a4e437e4f60b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:05.636903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:05.670527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43748c9a-b2d7-40aa-941a-6d50c7475503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:05.671591 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:05.672655 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43748c9a-b2d7-40aa-941a-6d50c7475503 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:05.673197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43748c9a-b2d7-40aa-941a-6d50c7475503 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:05.674331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43748c9a-b2d7-40aa-941a-6d50c7475503 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:05.675142 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43748c9a-b2d7-40aa-941a-6d50c7475503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:05.675908 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43748c9a-b2d7-40aa-941a-6d50c7475503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:05.677710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43748c9a-b2d7-40aa-941a-6d50c7475503 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:05.695525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 375/751] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:05.696972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43748c9a-b2d7-40aa-941a-6d50c7475503 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:05.743967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c40c9ae-3888-419f-a0e5-a4e437e4f60b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:05.792928 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ecc946-0ffb-4a27-aa0d-91ac23813f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:05.794224 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecc946-0ffb-4a27-aa0d-91ac23813f53 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:05.794927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecc946-0ffb-4a27-aa0d-91ac23813f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:05.795376 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ecc946-0ffb-4a27-aa0d-91ac23813f53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:05.811825 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ecc946-0ffb-4a27-aa0d-91ac23813f53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:05.814328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ecc946-0ffb-4a27-aa0d-91ac23813f53 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:05.817720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43748c9a-b2d7-40aa-941a-6d50c7475503 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b6ecc946-0ffb-4a27-aa0d-91ac23813f53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:05.835505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:05.836696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:05.837777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:05.837777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:05.837777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:05.839841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 377/752] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:05.840759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ecc946-0ffb-4a27-aa0d-91ac23813f53 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:05.859317 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c53b4f-1ae3-47aa-bbc2-c4cf1a6999ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:05.860106 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c53b4f-1ae3-47aa-bbc2-c4cf1a6999ed None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:05.860952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c53b4f-1ae3-47aa-bbc2-c4cf1a6999ed None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:05.860952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c53b4f-1ae3-47aa-bbc2-c4cf1a6999ed None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:05.861750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c53b4f-1ae3-47aa-bbc2-c4cf1a6999ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:05.862155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c53b4f-1ae3-47aa-bbc2-c4cf1a6999ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:05.863899 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c53b4f-1ae3-47aa-bbc2-c4cf1a6999ed None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:05.886936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 376/753] 10.222.0.67 () {56 vars in 1149 bytes} [Mon Sep 26 17:45:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:05.887955 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c53b4f-1ae3-47aa-bbc2-c4cf1a6999ed None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:05.930675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:05.960808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:05.972848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ecc946-0ffb-4a27-aa0d-91ac23813f53 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:06.019595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dedf9-0b93-4e86-96ba-1b4844429bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:06.020423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dedf9-0b93-4e86-96ba-1b4844429bef None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.020802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dedf9-0b93-4e86-96ba-1b4844429bef None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.021174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57dedf9-0b93-4e86-96ba-1b4844429bef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.028660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57dedf9-0b93-4e86-96ba-1b4844429bef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:06.030492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57dedf9-0b93-4e86-96ba-1b4844429bef None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:06.034534 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c53b4f-1ae3-47aa-bbc2-c4cf1a6999ed None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:06.043248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b57dedf9-0b93-4e86-96ba-1b4844429bef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:06.043248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:06.043248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:06.043248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:06.043248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:06.043248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:06.043248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:06.043248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:06.043248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:06.043248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:06.043248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:06.043248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:06.043248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:06.043248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:06.043248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:06.044364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:06.045438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:06.045438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:06.045438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:06.045438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:06.045438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:06.045438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:06.045438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:06.045438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:06.047447 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 378/754] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:06.048981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dedf9-0b93-4e86-96ba-1b4844429bef None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.086901 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0f9e9-b562-495d-b75e-a089ac0043dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:06.088425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0f9e9-b562-495d-b75e-a089ac0043dc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.088965 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0f9e9-b562-495d-b75e-a089ac0043dc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.089388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0f9e9-b562-495d-b75e-a089ac0043dc None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.090212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0f9e9-b562-495d-b75e-a089ac0043dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:06.090700 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0f9e9-b562-495d-b75e-a089ac0043dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:06.093718 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0f9e9-b562-495d-b75e-a089ac0043dc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:06.112299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 377/755] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:06.113422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0f9e9-b562-495d-b75e-a089ac0043dc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.169179 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57dedf9-0b93-4e86-96ba-1b4844429bef None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:06.207234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cce2ce-bdd4-41b3-8f51-0489f81e34f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:06.207881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cce2ce-bdd4-41b3-8f51-0489f81e34f0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.208160 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cce2ce-bdd4-41b3-8f51-0489f81e34f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.208436 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cce2ce-bdd4-41b3-8f51-0489f81e34f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.227291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:06.229860 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:06.240547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0f9e9-b562-495d-b75e-a089ac0043dc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:06.275746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d74860-b0bd-4314-9a1f-c75c85b74d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:06.276358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d74860-b0bd-4314-9a1f-c75c85b74d53 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.276636 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d74860-b0bd-4314-9a1f-c75c85b74d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.276905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d74860-b0bd-4314-9a1f-c75c85b74d53 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.277586 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d74860-b0bd-4314-9a1f-c75c85b74d53 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:06.277888 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d74860-b0bd-4314-9a1f-c75c85b74d53 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:06.279370 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d74860-b0bd-4314-9a1f-c75c85b74d53 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:06.280885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cce2ce-bdd4-41b3-8f51-0489f81e34f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:06.283097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cce2ce-bdd4-41b3-8f51-0489f81e34f0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:06.297903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 378/756] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:06.297903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d74860-b0bd-4314-9a1f-c75c85b74d53 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.393849 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 379/757] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:45:06.395126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ed6af-8657-4c68-a174-df7609f34ce1 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.395414 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ed6af-8657-4c68-a174-df7609f34ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.395710 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ed6af-8657-4c68-a174-df7609f34ce1 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.399836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 380/758] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:06.399836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ed6af-8657-4c68-a174-df7609f34ce1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.404757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d74860-b0bd-4314-9a1f-c75c85b74d53 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:06.439291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e133aa0-0c77-48c6-821c-e628a5f50ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:06.440023 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e133aa0-0c77-48c6-821c-e628a5f50ac4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.440304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e133aa0-0c77-48c6-821c-e628a5f50ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.440575 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e133aa0-0c77-48c6-821c-e628a5f50ac4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.447246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e133aa0-0c77-48c6-821c-e628a5f50ac4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:06.448808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e133aa0-0c77-48c6-821c-e628a5f50ac4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:06.454645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:06.460966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6e133aa0-0c77-48c6-821c-e628a5f50ac4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:06.460966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:06.460966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:06.460966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:06.460966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:06.460966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:06.460966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:06.460966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:06.460966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:06.460966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:06.460966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:06.460966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:06.460966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:06.460966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:06.460966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:06.467748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:06.468821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:06.468821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:06.468821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:06.468821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:06.468821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:06.468821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:06.468821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:06.468821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:06.468821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 379/759] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:06.468821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e133aa0-0c77-48c6-821c-e628a5f50ac4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.477920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:06.522468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ed6af-8657-4c68-a174-df7609f34ce1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:06.559000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cffc04-6f37-4d4d-93b6-a8b653d87035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:06.559779 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cffc04-6f37-4d4d-93b6-a8b653d87035 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.560042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cffc04-6f37-4d4d-93b6-a8b653d87035 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.560321 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cffc04-6f37-4d4d-93b6-a8b653d87035 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.561104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cffc04-6f37-4d4d-93b6-a8b653d87035 None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=7ba12b94717a4a40b55635192840d3c3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:06.561465 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cffc04-6f37-4d4d-93b6-a8b653d87035 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:06.562994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cffc04-6f37-4d4d-93b6-a8b653d87035 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:06.568845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 381/760] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:06 2022] GET /identity/v3/role_assignments?user.id=7ba12b94717a4a40b55635192840d3c3&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=07626899472b4e32adc0c71ce796a25d => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:06.568845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107f7e5-f1d4-41de-8bf6-5313ba154214 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.568845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107f7e5-f1d4-41de-8bf6-5313ba154214 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.568845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c107f7e5-f1d4-41de-8bf6-5313ba154214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.582327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e133aa0-0c77-48c6-821c-e628a5f50ac4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:06.591968 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c107f7e5-f1d4-41de-8bf6-5313ba154214 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:06.593375 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c107f7e5-f1d4-41de-8bf6-5313ba154214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:06.628299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce2015-b7e7-4635-bead-ca33fd594274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:06.629325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce2015-b7e7-4635-bead-ca33fd594274 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.629613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce2015-b7e7-4635-bead-ca33fd594274 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.630172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dce2015-b7e7-4635-bead-ca33fd594274 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.647083 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce2015-b7e7-4635-bead-ca33fd594274 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:06.649043 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dce2015-b7e7-4635-bead-ca33fd594274 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6dce2015-b7e7-4635-bead-ca33fd594274 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:06.668894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:06.669878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:06.670736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:06.670736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:06.670736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:06.675503 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 380/761] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:06.676613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce2015-b7e7-4635-bead-ca33fd594274 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.677558 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 382/762] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:06.678476 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107f7e5-f1d4-41de-8bf6-5313ba154214 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.747142 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:06.749159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:06.793021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dce2015-b7e7-4635-bead-ca33fd594274 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:06.831637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c107f7e5-f1d4-41de-8bf6-5313ba154214 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:06.834994 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24398da6-eb5a-4398-8bbe-40853e6e360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:06.834994 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24398da6-eb5a-4398-8bbe-40853e6e360b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.835225 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24398da6-eb5a-4398-8bbe-40853e6e360b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.835469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24398da6-eb5a-4398-8bbe-40853e6e360b None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.836196 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24398da6-eb5a-4398-8bbe-40853e6e360b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:06.836543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24398da6-eb5a-4398-8bbe-40853e6e360b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:06.837990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24398da6-eb5a-4398-8bbe-40853e6e360b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:06.860872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 381/763] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:06.866619 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24398da6-eb5a-4398-8bbe-40853e6e360b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.887533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d6320-af17-4806-a609-4ad9953c8547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:06.888393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d6320-af17-4806-a609-4ad9953c8547 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:06.889071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d6320-af17-4806-a609-4ad9953c8547 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:06.889436 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43d6320-af17-4806-a609-4ad9953c8547 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:06.890176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d6320-af17-4806-a609-4ad9953c8547 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:06.890577 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d6320-af17-4806-a609-4ad9953c8547 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:06.892915 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43d6320-af17-4806-a609-4ad9953c8547 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:06.908767 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 383/764] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:06.914148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d6320-af17-4806-a609-4ad9953c8547 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:06.984824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24398da6-eb5a-4398-8bbe-40853e6e360b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.007797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:07.025301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5412b-dc89-415e-b408-b639fc33b846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.025977 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5412b-dc89-415e-b408-b639fc33b846 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.026261 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5412b-dc89-415e-b408-b639fc33b846 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.026735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e5412b-dc89-415e-b408-b639fc33b846 None admin] PATH_INFO: `/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/b9ed29a3f6be4d64bb3751f9e95eb56b/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.029815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5412b-dc89-415e-b408-b639fc33b846 None admin] RBAC: Authorizing `identity:create_grant(project_id=71c96e5caeb44927bc21504fe54f0f98, user_id=b9ed29a3f6be4d64bb3751f9e95eb56b, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.031255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e5412b-dc89-415e-b408-b639fc33b846 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.061485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43d6320-af17-4806-a609-4ad9953c8547 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.072004 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:07.098743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190ae9e-d2c5-464d-a121-293ec7fb9f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.099641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190ae9e-d2c5-464d-a121-293ec7fb9f11 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.100133 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190ae9e-d2c5-464d-a121-293ec7fb9f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.100406 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190ae9e-d2c5-464d-a121-293ec7fb9f11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.107536 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190ae9e-d2c5-464d-a121-293ec7fb9f11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.108988 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190ae9e-d2c5-464d-a121-293ec7fb9f11 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.124213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2190ae9e-d2c5-464d-a121-293ec7fb9f11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:07.124213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:07.124213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:07.124213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:07.124213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:07.124213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:07.124213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:07.124213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:07.124213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:07.124213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:07.124213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:07.124213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:07.124213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:07.124213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:07.124213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:07.125197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:07.126778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:07.126778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:07.126778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:07.126778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:07.126778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:07.126778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:07.126778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:07.126778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:07.127212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 382/765] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:45:06 2022] PUT /identity/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/b9ed29a3f6be4d64bb3751f9e95eb56b/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:07.127212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5412b-dc89-415e-b408-b639fc33b846 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.129361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 384/766] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:07.137002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190ae9e-d2c5-464d-a121-293ec7fb9f11 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.310481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e5412b-dc89-415e-b408-b639fc33b846 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.326748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190ae9e-d2c5-464d-a121-293ec7fb9f11 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.358659 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a432cf-8bfb-4512-ab8b-a3ed438239b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.360815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a432cf-8bfb-4512-ab8b-a3ed438239b2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.361522 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a432cf-8bfb-4512-ab8b-a3ed438239b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.362043 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a432cf-8bfb-4512-ab8b-a3ed438239b2 None admin] PATH_INFO: `/v3/services/network` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.363026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a432cf-8bfb-4512-ab8b-a3ed438239b2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.365358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a432cf-8bfb-4512-ab8b-a3ed438239b2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.368890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c75c9f-7f9a-4c7c-8f0f-338bc1399d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.369726 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c75c9f-7f9a-4c7c-8f0f-338bc1399d48 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.369889 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c75c9f-7f9a-4c7c-8f0f-338bc1399d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.370163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c75c9f-7f9a-4c7c-8f0f-338bc1399d48 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.370848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c75c9f-7f9a-4c7c-8f0f-338bc1399d48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:07.371167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c75c9f-7f9a-4c7c-8f0f-338bc1399d48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.372569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c75c9f-7f9a-4c7c-8f0f-338bc1399d48 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d6a432cf-8bfb-4512-ab8b-a3ed438239b2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:07.378049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:07.379236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:07.380297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:07.380297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:07.380297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:07.380297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:07.380297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:07.380297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:07.380297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:07.380297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:07.380297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:07.380297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:07.380297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:07.380297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:07.380297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:45:07.380297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:07.383835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 383/767] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/services/network => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:07.385184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d72f62-1c7e-4829-b216-81699a7e1a90 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.385593 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d72f62-1c7e-4829-b216-81699a7e1a90 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.386055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d72f62-1c7e-4829-b216-81699a7e1a90 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.387671 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 384/768] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:07.390263 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d72f62-1c7e-4829-b216-81699a7e1a90 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.392113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 385/769] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:07.396491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342901e-41f0-487d-a1b7-39828fa5fd64 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.396876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342901e-41f0-487d-a1b7-39828fa5fd64 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.397249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342901e-41f0-487d-a1b7-39828fa5fd64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.422571 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d342901e-41f0-487d-a1b7-39828fa5fd64 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:07.424616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d342901e-41f0-487d-a1b7-39828fa5fd64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:07.446360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:07.457080 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:07.514499 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 386/770] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:07.515862 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342901e-41f0-487d-a1b7-39828fa5fd64 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.534647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d72f62-1c7e-4829-b216-81699a7e1a90 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.573816 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed871d9-bca0-4e0a-9498-bc28666d971f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.574748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed871d9-bca0-4e0a-9498-bc28666d971f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.574748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed871d9-bca0-4e0a-9498-bc28666d971f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.575203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed871d9-bca0-4e0a-9498-bc28666d971f None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.576065 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed871d9-bca0-4e0a-9498-bc28666d971f None admin] RBAC: Adding query filter params (name=network) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:07.576475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed871d9-bca0-4e0a-9498-bc28666d971f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.578049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed871d9-bca0-4e0a-9498-bc28666d971f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.584019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 385/771] 10.222.0.67 () {56 vars in 1112 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:07.590001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed871d9-bca0-4e0a-9498-bc28666d971f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.645456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342901e-41f0-487d-a1b7-39828fa5fd64 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.682278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a446b5-12c3-4a55-87aa-d8ec724d2dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.683102 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a446b5-12c3-4a55-87aa-d8ec724d2dce None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.683484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a446b5-12c3-4a55-87aa-d8ec724d2dce None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.683899 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a446b5-12c3-4a55-87aa-d8ec724d2dce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.684736 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a446b5-12c3-4a55-87aa-d8ec724d2dce None admin] RBAC: Adding query filter params (role.id=90322db354f94f03b32ec088922baf2e, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=1d201727ffae42c4bd704d26c03c5d9b) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:07.685144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a446b5-12c3-4a55-87aa-d8ec724d2dce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.686660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a446b5-12c3-4a55-87aa-d8ec724d2dce None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.694327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 387/772] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/role_assignments?user.id=1d201727ffae42c4bd704d26c03c5d9b&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=90322db354f94f03b32ec088922baf2e => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:07.708172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed871d9-bca0-4e0a-9498-bc28666d971f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.719346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:07.747728 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c109e-8cad-4ce4-9e0f-4b223821c556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.748610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c109e-8cad-4ce4-9e0f-4b223821c556 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.749071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c109e-8cad-4ce4-9e0f-4b223821c556 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.749484 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c109e-8cad-4ce4-9e0f-4b223821c556 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.750271 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c109e-8cad-4ce4-9e0f-4b223821c556 None admin] RBAC: Adding query filter params (type=network) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:07.750719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c109e-8cad-4ce4-9e0f-4b223821c556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.753006 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c109e-8cad-4ce4-9e0f-4b223821c556 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.759314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 386/773] 10.222.0.67 () {56 vars in 1112 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:07.766031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a446b5-12c3-4a55-87aa-d8ec724d2dce None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.808297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:07.901422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a446b5-12c3-4a55-87aa-d8ec724d2dce None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:07.924761 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c109e-8cad-4ce4-9e0f-4b223821c556 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:07.946422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5c6bb-dc7c-4099-b1b6-3802b83d4fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:07.947119 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5c6bb-dc7c-4099-b1b6-3802b83d4fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:07.947335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5c6bb-dc7c-4099-b1b6-3802b83d4fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:07.947609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5c6bb-dc7c-4099-b1b6-3802b83d4fe2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:07.948344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5c6bb-dc7c-4099-b1b6-3802b83d4fe2 None admin] RBAC: Adding query filter params (interface=public, service_id=fd1955deb2bc48d0985ae286ea245e58, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:07.948657 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5c6bb-dc7c-4099-b1b6-3802b83d4fe2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:07.950062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5c6bb-dc7c-4099-b1b6-3802b83d4fe2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:07.970592 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 388/774] 10.222.0.67 () {56 vars in 1251 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/endpoints?service_id=fd1955deb2bc48d0985ae286ea245e58&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:07.976365 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5c6bb-dc7c-4099-b1b6-3802b83d4fe2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:08.100840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:08.101838 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c109e-8cad-4ce4-9e0f-4b223821c556 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:08.121325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5c6bb-dc7c-4099-b1b6-3802b83d4fe2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:08.167939 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70499db-e419-419f-8789-0ab747a65c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:08.168128 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004e3c7-2bcc-4f47-a5d9-b3f32ccc5ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:08.168524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70499db-e419-419f-8789-0ab747a65c37 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:08.168648 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004e3c7-2bcc-4f47-a5d9-b3f32ccc5ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:08.168808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70499db-e419-419f-8789-0ab747a65c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:08.168903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004e3c7-2bcc-4f47-a5d9-b3f32ccc5ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:08.169109 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70499db-e419-419f-8789-0ab747a65c37 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:08.169211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c004e3c7-2bcc-4f47-a5d9-b3f32ccc5ca6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:08.169718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70499db-e419-419f-8789-0ab747a65c37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:08.169835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004e3c7-2bcc-4f47-a5d9-b3f32ccc5ca6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:08.171094 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70499db-e419-419f-8789-0ab747a65c37 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:08.171203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c004e3c7-2bcc-4f47-a5d9-b3f32ccc5ca6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:08.185439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 389/775] 10.222.0.67 () {56 vars in 1087 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/services => generated 772 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c004e3c7-2bcc-4f47-a5d9-b3f32ccc5ca6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:08.189209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:08.190177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:08.191012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:08.191012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:08.191012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:08.191012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:08.191012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:08.191012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:08.191012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:08.191012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:08.191012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:08.191012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:08.191012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:08.191012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:08.191012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:45:08.191012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:08.191012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 387/776] 10.222.0.67 () {56 vars in 1129 bytes} [Mon Sep 26 17:45:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:08.198026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70499db-e419-419f-8789-0ab747a65c37 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:08.352208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70499db-e419-419f-8789-0ab747a65c37 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:08.413546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a741fb7-60a1-4900-b4e1-d15b92029639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:08.414179 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a741fb7-60a1-4900-b4e1-d15b92029639 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:08.414462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a741fb7-60a1-4900-b4e1-d15b92029639 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:08.414739 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a741fb7-60a1-4900-b4e1-d15b92029639 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:08.415388 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a741fb7-60a1-4900-b4e1-d15b92029639 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:08.415715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a741fb7-60a1-4900-b4e1-d15b92029639 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:08.417134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a741fb7-60a1-4900-b4e1-d15b92029639 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:08.428073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 390/777] 10.222.0.67 () {56 vars in 1124 bytes} [Mon Sep 26 17:45:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:08.429922 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004e3c7-2bcc-4f47-a5d9-b3f32ccc5ca6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:08.485738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:08.570116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c004e3c7-2bcc-4f47-a5d9-b3f32ccc5ca6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:08.621271 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8b543-2a11-472b-b8d9-88008bb69952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:08.621882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8b543-2a11-472b-b8d9-88008bb69952 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:08.622156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8b543-2a11-472b-b8d9-88008bb69952 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:08.622425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8b543-2a11-472b-b8d9-88008bb69952 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:08.623051 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8b543-2a11-472b-b8d9-88008bb69952 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:08.623346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8b543-2a11-472b-b8d9-88008bb69952 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:08.624731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8b543-2a11-472b-b8d9-88008bb69952 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:08.636039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 388/778] 10.222.0.67 () {56 vars in 1124 bytes} [Mon Sep 26 17:45:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:08.642069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a741fb7-60a1-4900-b4e1-d15b92029639 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:08.660671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:08.768017 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a741fb7-60a1-4900-b4e1-d15b92029639 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:08.815691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bce235-43b0-4eaa-b074-1ffed4e02392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:08.816308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bce235-43b0-4eaa-b074-1ffed4e02392 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:08.816585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bce235-43b0-4eaa-b074-1ffed4e02392 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:08.816853 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bce235-43b0-4eaa-b074-1ffed4e02392 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:08.817548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bce235-43b0-4eaa-b074-1ffed4e02392 None admin] RBAC: Adding query filter params (interface=public, service_id=5852e6158a9e4abb8c1893d01fc86cb6, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:08.817845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bce235-43b0-4eaa-b074-1ffed4e02392 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:08.819206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bce235-43b0-4eaa-b074-1ffed4e02392 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:08.824908 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 391/779] 10.222.0.67 () {56 vars in 1251 bytes} [Mon Sep 26 17:45:08 2022] GET /identity/v3/endpoints?service_id=5852e6158a9e4abb8c1893d01fc86cb6&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:08.831305 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8b543-2a11-472b-b8d9-88008bb69952 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:08.974271 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:08.980246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8b543-2a11-472b-b8d9-88008bb69952 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:09.031288 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d585a2-6465-47e5-9ff5-e9f3668612e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:09.031840 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d585a2-6465-47e5-9ff5-e9f3668612e7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:09.032286 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d585a2-6465-47e5-9ff5-e9f3668612e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:09.032905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d585a2-6465-47e5-9ff5-e9f3668612e7 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:09.033489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d585a2-6465-47e5-9ff5-e9f3668612e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:09.035150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d585a2-6465-47e5-9ff5-e9f3668612e7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:09.045252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 389/780] 10.222.0.67 () {56 vars in 1087 bytes} [Mon Sep 26 17:45:08 2022] GET /identity/v3/services => generated 772 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:09.498270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74659a9a-0d4e-4015-a24d-3f9f0643a36c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:09.498788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74659a9a-0d4e-4015-a24d-3f9f0643a36c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:09.499293 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74659a9a-0d4e-4015-a24d-3f9f0643a36c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:09.501904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 392/781] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:09.506809 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b5828-c4bd-4621-bc0e-d3d47886894b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:09.507101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b5828-c4bd-4621-bc0e-d3d47886894b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:09.507383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b5828-c4bd-4621-bc0e-d3d47886894b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:09.528605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-447b5828-c4bd-4621-bc0e-d3d47886894b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:09.530199 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-447b5828-c4bd-4621-bc0e-d3d47886894b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:09.624536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 390/782] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:09.737425 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b25c1-e9be-4141-a558-c53d0861550d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:09.737956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b25c1-e9be-4141-a558-c53d0861550d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:09.737956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b25c1-e9be-4141-a558-c53d0861550d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:09.739520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 393/783] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:09.745014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32e113-9dbd-49ba-b03e-1e88354741e6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:09.745537 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32e113-9dbd-49ba-b03e-1e88354741e6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:09.746011 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32e113-9dbd-49ba-b03e-1e88354741e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:09.770421 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1e32e113-9dbd-49ba-b03e-1e88354741e6 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:09.771965 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1e32e113-9dbd-49ba-b03e-1e88354741e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:09.856996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 391/784] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:09.952884 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b25c1-e9be-4141-a558-c53d0861550d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:10.072625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:10.105506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b25c1-e9be-4141-a558-c53d0861550d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:10.151754 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa83da-2b80-4c89-a7c8-3b76b55ef696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:10.152469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa83da-2b80-4c89-a7c8-3b76b55ef696 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.152763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa83da-2b80-4c89-a7c8-3b76b55ef696 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.153041 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa83da-2b80-4c89-a7c8-3b76b55ef696 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.170644 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30aa83da-2b80-4c89-a7c8-3b76b55ef696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-30aa83da-2b80-4c89-a7c8-3b76b55ef696 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:10.177059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:10.178045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:10.178847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:10.178847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:10.178847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:10.178847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:10.178847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:10.178847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:10.178847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:10.178847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:10.178847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:10.178847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:10.178847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:10.178847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:10.178847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 394/785] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:10.184804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32e113-9dbd-49ba-b03e-1e88354741e6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:10.235901 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:10.290599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa83da-2b80-4c89-a7c8-3b76b55ef696 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:10.342897 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e32e113-9dbd-49ba-b03e-1e88354741e6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:10.388551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26743d46-5f02-4bf6-a22c-be18f83687ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:10.389685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26743d46-5f02-4bf6-a22c-be18f83687ce None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.389968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26743d46-5f02-4bf6-a22c-be18f83687ce None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.390720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26743d46-5f02-4bf6-a22c-be18f83687ce None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.392101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26743d46-5f02-4bf6-a22c-be18f83687ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:10.392519 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26743d46-5f02-4bf6-a22c-be18f83687ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:10.393955 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26743d46-5f02-4bf6-a22c-be18f83687ce None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:10.408748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 392/786] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:10.408748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f0a37-857e-4f22-9124-5a1f2101bfc4 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.408748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f0a37-857e-4f22-9124-5a1f2101bfc4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.408748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06f0a37-857e-4f22-9124-5a1f2101bfc4 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.408748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 393/787] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:10.414134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a0fdc-4890-4db8-9be0-7eea8d596997 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.414427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a0fdc-4890-4db8-9be0-7eea8d596997 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.414709 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0a0fdc-4890-4db8-9be0-7eea8d596997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.425746 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa83da-2b80-4c89-a7c8-3b76b55ef696 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:10.433201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9c0a0fdc-4890-4db8-9be0-7eea8d596997 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:10.434580 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9c0a0fdc-4890-4db8-9be0-7eea8d596997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:10.472026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54de122-c923-4f1a-8e4d-42db35feeb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:10.473172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54de122-c923-4f1a-8e4d-42db35feeb6e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.474537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54de122-c923-4f1a-8e4d-42db35feeb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.474976 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54de122-c923-4f1a-8e4d-42db35feeb6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.477041 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:10.490425 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54de122-c923-4f1a-8e4d-42db35feeb6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a54de122-c923-4f1a-8e4d-42db35feeb6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:10.496747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:10.503014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:10.504204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:10.504204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:10.504204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:10.504204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:10.504204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:10.504204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:10.504204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:10.504204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:10.504204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:10.504204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:10.504204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:10.504204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:10.504204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 395/788] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:10.504204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54de122-c923-4f1a-8e4d-42db35feeb6e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:10.520459 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 394/789] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:10.521463 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a0fdc-4890-4db8-9be0-7eea8d596997 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:10.562771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:10.631385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54de122-c923-4f1a-8e4d-42db35feeb6e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:10.671236 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2558a4-c9e0-4107-87e3-3ca043bd1290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:10.671895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2558a4-c9e0-4107-87e3-3ca043bd1290 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.672177 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2558a4-c9e0-4107-87e3-3ca043bd1290 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.673518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2558a4-c9e0-4107-87e3-3ca043bd1290 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.684839 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2558a4-c9e0-4107-87e3-3ca043bd1290 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:10.686969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2558a4-c9e0-4107-87e3-3ca043bd1290 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8c2558a4-c9e0-4107-87e3-3ca043bd1290 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:10.704358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:10.705512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:10.706542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:10.706542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:10.706542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:10.712968 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 396/790] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:10.714630 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2558a4-c9e0-4107-87e3-3ca043bd1290 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:10.764783 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:10.781578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0a0fdc-4890-4db8-9be0-7eea8d596997 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:10.846100 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2558a4-c9e0-4107-87e3-3ca043bd1290 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:10.877472 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8789037-eb47-4b22-b917-57153873cb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:10.878273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8789037-eb47-4b22-b917-57153873cb35 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.878642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8789037-eb47-4b22-b917-57153873cb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.879010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8789037-eb47-4b22-b917-57153873cb35 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.879817 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8789037-eb47-4b22-b917-57153873cb35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:10.880218 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8789037-eb47-4b22-b917-57153873cb35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:10.881721 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8789037-eb47-4b22-b917-57153873cb35 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:10.883500 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1c4bd-8ad6-49b8-bed3-7cbacc422b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:10.884352 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1c4bd-8ad6-49b8-bed3-7cbacc422b89 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:10.884816 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1c4bd-8ad6-49b8-bed3-7cbacc422b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:10.885295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1c4bd-8ad6-49b8-bed3-7cbacc422b89 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:10.886116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1c4bd-8ad6-49b8-bed3-7cbacc422b89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:10.886611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1c4bd-8ad6-49b8-bed3-7cbacc422b89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:10.888210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1c4bd-8ad6-49b8-bed3-7cbacc422b89 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:10.899210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 395/791] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:10.900358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8789037-eb47-4b22-b917-57153873cb35 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:10.904004 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 397/792] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:10.906302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1c4bd-8ad6-49b8-bed3-7cbacc422b89 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.007621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:11.030181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8789037-eb47-4b22-b917-57153873cb35 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.042996 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1c4bd-8ad6-49b8-bed3-7cbacc422b89 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.053101 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:11.066827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa801661-90ad-4ffc-b0c8-f780fa1e9aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.067517 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa801661-90ad-4ffc-b0c8-f780fa1e9aab None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.067888 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa801661-90ad-4ffc-b0c8-f780fa1e9aab None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.068168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa801661-90ad-4ffc-b0c8-f780fa1e9aab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.079383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa801661-90ad-4ffc-b0c8-f780fa1e9aab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fa801661-90ad-4ffc-b0c8-f780fa1e9aab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:11.084736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:11.085805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:11.086606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:11.086606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:11.086606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:11.086606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:11.086606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:11.086606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:11.086606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:11.086606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:11.086606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:11.086606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:11.086606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:11.086606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:11.086606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 396/793] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:45:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:11.086606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa801661-90ad-4ffc-b0c8-f780fa1e9aab None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.088246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca341b-b1a2-43a9-81e7-fe3dd5e2226b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.089070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca341b-b1a2-43a9-81e7-fe3dd5e2226b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.089502 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca341b-b1a2-43a9-81e7-fe3dd5e2226b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.089924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca341b-b1a2-43a9-81e7-fe3dd5e2226b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.103176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca341b-b1a2-43a9-81e7-fe3dd5e2226b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.104655 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca341b-b1a2-43a9-81e7-fe3dd5e2226b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-9aca341b-b1a2-43a9-81e7-fe3dd5e2226b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:11.119203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:11.120176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:11.121084 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:11.121084 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:11.121084 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:11.121084 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 398/794] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:11.121585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ee1a5-f984-4875-9806-a8ec4fc2978f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.121709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ee1a5-f984-4875-9806-a8ec4fc2978f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.121974 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ee1a5-f984-4875-9806-a8ec4fc2978f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.123342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 399/795] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:11.127911 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ee1a5-f984-4875-9806-a8ec4fc2978f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.201362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa801661-90ad-4ffc-b0c8-f780fa1e9aab None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.236337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ee1a5-f984-4875-9806-a8ec4fc2978f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.237126 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08296d-a286-4318-950b-ff23b4ff5663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.237747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08296d-a286-4318-950b-ff23b4ff5663 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.238024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08296d-a286-4318-950b-ff23b4ff5663 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.238314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08296d-a286-4318-950b-ff23b4ff5663 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.245594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08296d-a286-4318-950b-ff23b4ff5663 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.247020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08296d-a286-4318-950b-ff23b4ff5663 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.248750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4a08296d-a286-4318-950b-ff23b4ff5663 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:11.258561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:11.265527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:11.267824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:11.267824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:11.267824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:11.267824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:11.267824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:11.267824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:11.267824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:11.267824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:11.267824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:45:11.267824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:11.267824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 397/796] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:11.267824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08296d-a286-4318-950b-ff23b4ff5663 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.279949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08cb724-6251-4a63-901b-81212d0486fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.280559 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cb724-6251-4a63-901b-81212d0486fd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.280836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cb724-6251-4a63-901b-81212d0486fd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.281109 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08cb724-6251-4a63-901b-81212d0486fd None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.281756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cb724-6251-4a63-901b-81212d0486fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:11.282678 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cb724-6251-4a63-901b-81212d0486fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.284155 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08cb724-6251-4a63-901b-81212d0486fd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.298407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 400/797] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:11.299700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bf2c4-a726-4673-ae5c-c422ea504717 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.299981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bf2c4-a726-4673-ae5c-c422ea504717 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.300252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bf2c4-a726-4673-ae5c-c422ea504717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.318691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8d3bf2c4-a726-4673-ae5c-c422ea504717 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:11.319920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8d3bf2c4-a726-4673-ae5c-c422ea504717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:11.335559 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:11.373385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08296d-a286-4318-950b-ff23b4ff5663 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.397502 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 401/798] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:11.398253 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bf2c4-a726-4673-ae5c-c422ea504717 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.425540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c430c2-78e3-4beb-a6f4-80b4e605a936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.426159 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c430c2-78e3-4beb-a6f4-80b4e605a936 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.426437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c430c2-78e3-4beb-a6f4-80b4e605a936 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.426706 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c430c2-78e3-4beb-a6f4-80b4e605a936 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.427342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c430c2-78e3-4beb-a6f4-80b4e605a936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:11.427640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c430c2-78e3-4beb-a6f4-80b4e605a936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.429040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c430c2-78e3-4beb-a6f4-80b4e605a936 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.448509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 398/799] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:11.449396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c430c2-78e3-4beb-a6f4-80b4e605a936 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.514956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:11.530430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bf2c4-a726-4673-ae5c-c422ea504717 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.559718 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c430c2-78e3-4beb-a6f4-80b4e605a936 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.583323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e05b5-c2cc-49ee-b9f7-1a180a88b094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.584572 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e05b5-c2cc-49ee-b9f7-1a180a88b094 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.585231 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e05b5-c2cc-49ee-b9f7-1a180a88b094 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.585920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e05b5-c2cc-49ee-b9f7-1a180a88b094 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.587411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e05b5-c2cc-49ee-b9f7-1a180a88b094 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:11.588034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e05b5-c2cc-49ee-b9f7-1a180a88b094 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.590328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e05b5-c2cc-49ee-b9f7-1a180a88b094 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.598498 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758eb123-7b09-4cd7-b2e1-40188814926c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.599220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758eb123-7b09-4cd7-b2e1-40188814926c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.599572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758eb123-7b09-4cd7-b2e1-40188814926c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.599958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758eb123-7b09-4cd7-b2e1-40188814926c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.611081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758eb123-7b09-4cd7-b2e1-40188814926c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.612692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758eb123-7b09-4cd7-b2e1-40188814926c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.615066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 402/800] 10.222.0.67 () {56 vars in 1135 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:11.616123 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e05b5-c2cc-49ee-b9f7-1a180a88b094 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.629889 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-758eb123-7b09-4cd7-b2e1-40188814926c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:11.634219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:11.635240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:11.636369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:11.636369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:11.636369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:11.640342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 399/801] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:11.641338 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758eb123-7b09-4cd7-b2e1-40188814926c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.735945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e05b5-c2cc-49ee-b9f7-1a180a88b094 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.752366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758eb123-7b09-4cd7-b2e1-40188814926c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.763457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:11.772087 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece083a-4eec-42ac-8b9c-5c63d091fa51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.772681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece083a-4eec-42ac-8b9c-5c63d091fa51 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.772927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece083a-4eec-42ac-8b9c-5c63d091fa51 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.773194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece083a-4eec-42ac-8b9c-5c63d091fa51 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.780579 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece083a-4eec-42ac-8b9c-5c63d091fa51 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.783200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece083a-4eec-42ac-8b9c-5c63d091fa51 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.790086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1cbe3-b3c3-4c4b-a1df-74701c7edc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.791006 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1cbe3-b3c3-4c4b-a1df-74701c7edc98 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.791288 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1cbe3-b3c3-4c4b-a1df-74701c7edc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.791564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd1cbe3-b3c3-4c4b-a1df-74701c7edc98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6ece083a-4eec-42ac-8b9c-5c63d091fa51 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:11.796205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:11.797062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:11.798820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:11.798820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:11.798820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:11.798820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:11.798820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:11.798820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:11.798820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:11.798820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:11.798820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:45:11.798820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:11.802011 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 403/802] 10.222.0.67 () {56 vars in 1093 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:11.802925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece083a-4eec-42ac-8b9c-5c63d091fa51 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.807483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd1cbe3-b3c3-4c4b-a1df-74701c7edc98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.809245 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd1cbe3-b3c3-4c4b-a1df-74701c7edc98 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fcd1cbe3-b3c3-4c4b-a1df-74701c7edc98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:11.829982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:11.831010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:11.831911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:11.831911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:11.831911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:11.837515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 400/803] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:11.838498 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1cbe3-b3c3-4c4b-a1df-74701c7edc98 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.878855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:11.915607 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece083a-4eec-42ac-8b9c-5c63d091fa51 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.948606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd1cbe3-b3c3-4c4b-a1df-74701c7edc98 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:11.952359 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edfbcd-3650-4648-b1cd-de429dc8882c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.952966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edfbcd-3650-4648-b1cd-de429dc8882c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.953245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edfbcd-3650-4648-b1cd-de429dc8882c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.953515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9edfbcd-3650-4648-b1cd-de429dc8882c None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.954709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edfbcd-3650-4648-b1cd-de429dc8882c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:11.955146 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edfbcd-3650-4648-b1cd-de429dc8882c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.956537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9edfbcd-3650-4648-b1cd-de429dc8882c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.967906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 404/804] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:11.968870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edfbcd-3650-4648-b1cd-de429dc8882c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:11.987111 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77fcba-42b0-477a-bd4b-27e36f09a0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:11.987793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77fcba-42b0-477a-bd4b-27e36f09a0bf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:11.988085 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77fcba-42b0-477a-bd4b-27e36f09a0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:11.988359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d77fcba-42b0-477a-bd4b-27e36f09a0bf None admin] PATH_INFO: `/v3/services/network` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:11.989042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77fcba-42b0-477a-bd4b-27e36f09a0bf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:11.990426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d77fcba-42b0-477a-bd4b-27e36f09a0bf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7d77fcba-42b0-477a-bd4b-27e36f09a0bf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:11.999267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:12.000269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:12.001164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:12.001164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:12.001164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:12.001164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:12.001164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:12.001164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:12.001164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:12.001164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:12.001164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:12.001164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:12.001164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:12.001164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:12.001164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 26 17:45:12.001164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:12.001164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 401/805] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:12.001960 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77fcba-42b0-477a-bd4b-27e36f09a0bf None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:12.029300 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:12.076577 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9edfbcd-3650-4648-b1cd-de429dc8882c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:12.102813 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d77fcba-42b0-477a-bd4b-27e36f09a0bf None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:12.104097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:12.111782 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35575353-0562-41da-821d-59fc7060a6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:12.112387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35575353-0562-41da-821d-59fc7060a6b8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.112668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35575353-0562-41da-821d-59fc7060a6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.112941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35575353-0562-41da-821d-59fc7060a6b8 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:12.113653 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35575353-0562-41da-821d-59fc7060a6b8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:12.113949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35575353-0562-41da-821d-59fc7060a6b8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:12.115370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35575353-0562-41da-821d-59fc7060a6b8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:12.133231 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 405/806] 10.222.0.67 () {56 vars in 1133 bytes} [Mon Sep 26 17:45:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:12.134342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099fd91-fa44-4517-b2ca-f7654d621212 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.134623 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099fd91-fa44-4517-b2ca-f7654d621212 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.134893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099fd91-fa44-4517-b2ca-f7654d621212 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:12.136346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 406/807] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:12.136763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099fd91-fa44-4517-b2ca-f7654d621212 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:12.143684 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51de3d-6871-42b3-941c-4ceead08a665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:12.144294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51de3d-6871-42b3-941c-4ceead08a665 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.144568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51de3d-6871-42b3-941c-4ceead08a665 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.144832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c51de3d-6871-42b3-941c-4ceead08a665 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:12.145466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51de3d-6871-42b3-941c-4ceead08a665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:12.145762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51de3d-6871-42b3-941c-4ceead08a665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:12.147156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c51de3d-6871-42b3-941c-4ceead08a665 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:12.159205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 402/808] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:12.160401 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51de3d-6871-42b3-941c-4ceead08a665 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:12.357999 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099fd91-fa44-4517-b2ca-f7654d621212 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:12.362425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:12.377681 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c51de3d-6871-42b3-941c-4ceead08a665 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:12.392353 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c8eea-da01-44ba-954d-5f347b3a7707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:12.393079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c8eea-da01-44ba-954d-5f347b3a7707 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.393347 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c8eea-da01-44ba-954d-5f347b3a7707 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.393609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0c8eea-da01-44ba-954d-5f347b3a7707 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:12.400813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c8eea-da01-44ba-954d-5f347b3a7707 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:12.402231 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0c8eea-da01-44ba-954d-5f347b3a7707 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:12.413542 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2f20f2-3e7b-424b-9c2f-2744e16ad04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5f0c8eea-da01-44ba-954d-5f347b3a7707 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:12.413709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:12.414814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:12.415916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:12.415916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:12.415916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:12.415916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:12.415916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:12.415916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:12.415916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:12.415916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:12.415916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:45:12.415916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:12.416440 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f20f2-3e7b-424b-9c2f-2744e16ad04f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.416440 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f20f2-3e7b-424b-9c2f-2744e16ad04f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.416440 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2f20f2-3e7b-424b-9c2f-2744e16ad04f None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:12.416620 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 407/809] 10.222.0.67 () {56 vars in 1117 bytes} [Mon Sep 26 17:45:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:12.416695 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f20f2-3e7b-424b-9c2f-2744e16ad04f None admin] RBAC: Adding query filter params (name=network) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:12.416695 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f20f2-3e7b-424b-9c2f-2744e16ad04f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:12.416815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c8eea-da01-44ba-954d-5f347b3a7707 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:12.416880 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2f20f2-3e7b-424b-9c2f-2744e16ad04f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:12.423327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 403/810] 10.222.0.67 () {56 vars in 1112 bytes} [Mon Sep 26 17:45:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:12.424583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5798d6-4ea1-4eae-b389-8a7034533a14 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.424862 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5798d6-4ea1-4eae-b389-8a7034533a14 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.425128 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5798d6-4ea1-4eae-b389-8a7034533a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:12.443021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9b5798d6-4ea1-4eae-b389-8a7034533a14 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:12.444228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9b5798d6-4ea1-4eae-b389-8a7034533a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:12.467171 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:12.509099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 404/811] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:12.509719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5798d6-4ea1-4eae-b389-8a7034533a14 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:12.522746 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0c8eea-da01-44ba-954d-5f347b3a7707 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:12.559668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32699fd0-dfdb-44d4-8f20-c73628c74876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:12.560290 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32699fd0-dfdb-44d4-8f20-c73628c74876 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.560575 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32699fd0-dfdb-44d4-8f20-c73628c74876 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.560869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32699fd0-dfdb-44d4-8f20-c73628c74876 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:12.596224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:12.616447 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5798d6-4ea1-4eae-b389-8a7034533a14 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:12.632995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32699fd0-dfdb-44d4-8f20-c73628c74876 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:12.634455 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32699fd0-dfdb-44d4-8f20-c73628c74876 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:12.654340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269237cb-e441-4d56-af9d-12af14b67294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:12.655199 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269237cb-e441-4d56-af9d-12af14b67294 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.655767 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269237cb-e441-4d56-af9d-12af14b67294 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.656036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269237cb-e441-4d56-af9d-12af14b67294 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:12.666109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269237cb-e441-4d56-af9d-12af14b67294 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:12.667684 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269237cb-e441-4d56-af9d-12af14b67294 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:12.679434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-269237cb-e441-4d56-af9d-12af14b67294 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:12.679434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:12.679434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:12.679434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:12.679434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:12.679434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:12.679434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:12.679434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:12.679434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:12.679434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:12.679434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:12.679434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:12.679434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:12.679434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:12.679434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:12.680475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:12.681364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:12.681364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:12.681364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:12.681364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:12.681364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:12.681364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:12.681364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:12.681364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:12.681364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 405/812] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:12.681837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269237cb-e441-4d56-af9d-12af14b67294 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:12.687320 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:12.744543 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 408/813] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:45:12.749805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32699fd0-dfdb-44d4-8f20-c73628c74876 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:12.786391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269237cb-e441-4d56-af9d-12af14b67294 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:12.819078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f790b-a835-4538-849c-a276a197cc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:12.819886 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f790b-a835-4538-849c-a276a197cc68 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.820009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f790b-a835-4538-849c-a276a197cc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.820214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41f790b-a835-4538-849c-a276a197cc68 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:12.820933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f790b-a835-4538-849c-a276a197cc68 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:12.821232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f790b-a835-4538-849c-a276a197cc68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:12.822644 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41f790b-a835-4538-849c-a276a197cc68 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:12.832728 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:12.841120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 406/814] 10.222.0.67 () {56 vars in 1149 bytes} [Mon Sep 26 17:45:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:12.841831 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f790b-a835-4538-849c-a276a197cc68 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:12.853537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32699fd0-dfdb-44d4-8f20-c73628c74876 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:12.887435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62994b99-08a3-47dd-971f-6bea5541c743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:12.888077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62994b99-08a3-47dd-971f-6bea5541c743 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.888357 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62994b99-08a3-47dd-971f-6bea5541c743 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.888889 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62994b99-08a3-47dd-971f-6bea5541c743 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:12.889513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62994b99-08a3-47dd-971f-6bea5541c743 None admin] RBAC: Adding query filter params (type=network) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:12.889811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62994b99-08a3-47dd-971f-6bea5541c743 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:12.891734 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62994b99-08a3-47dd-971f-6bea5541c743 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:12.896479 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 409/815] 10.222.0.67 () {56 vars in 1112 bytes} [Mon Sep 26 17:45:12 2022] GET /identity/v3/services?type=network => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:12.897258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62994b99-08a3-47dd-971f-6bea5541c743 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:12.909422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:12.949815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41f790b-a835-4538-849c-a276a197cc68 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:12.987848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb41b0b-dda0-47b4-80c3-73a4c3d2926b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:12.988453 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb41b0b-dda0-47b4-80c3-73a4c3d2926b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:12.988726 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb41b0b-dda0-47b4-80c3-73a4c3d2926b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:12.988992 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb41b0b-dda0-47b4-80c3-73a4c3d2926b None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:12.989669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb41b0b-dda0-47b4-80c3-73a4c3d2926b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:12.989966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb41b0b-dda0-47b4-80c3-73a4c3d2926b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:12.991559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb41b0b-dda0-47b4-80c3-73a4c3d2926b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.003801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 407/816] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:13.004581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb41b0b-dda0-47b4-80c3-73a4c3d2926b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:13.007471 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62994b99-08a3-47dd-971f-6bea5541c743 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:13.042415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192a919-c16a-42f4-afde-1c52eee8f67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:13.043048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192a919-c16a-42f4-afde-1c52eee8f67b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.043325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192a919-c16a-42f4-afde-1c52eee8f67b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.043595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192a919-c16a-42f4-afde-1c52eee8f67b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.053991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192a919-c16a-42f4-afde-1c52eee8f67b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.055415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0192a919-c16a-42f4-afde-1c52eee8f67b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0192a919-c16a-42f4-afde-1c52eee8f67b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:13.069470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:13.070491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:13.071871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:13.071871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:13.071871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:13.072094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:13.072192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 410/817] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:13.072192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192a919-c16a-42f4-afde-1c52eee8f67b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:13.115262 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb41b0b-dda0-47b4-80c3-73a4c3d2926b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:13.130139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:13.148678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293fa42-68e4-461b-820d-560c9e44c331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:13.149234 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293fa42-68e4-461b-820d-560c9e44c331 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.149511 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293fa42-68e4-461b-820d-560c9e44c331 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.149776 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0293fa42-68e4-461b-820d-560c9e44c331 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.150346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293fa42-68e4-461b-820d-560c9e44c331 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.151752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0293fa42-68e4-461b-820d-560c9e44c331 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0293fa42-68e4-461b-820d-560c9e44c331 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:13.161116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:13.162089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:13.163415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:13.163415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:13.163415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:13.163415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:13.163415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:13.163415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:13.163415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:13.163415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:13.163415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:13.163415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:13.163415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:13.163415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:13.163415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 26 17:45:13.163415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:13.163415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 408/818] 10.222.0.67 () {56 vars in 1129 bytes} [Mon Sep 26 17:45:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:13.164531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293fa42-68e4-461b-820d-560c9e44c331 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:13.174174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192a919-c16a-42f4-afde-1c52eee8f67b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:13.207245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ce708-c2da-43ac-a0b9-1b58d70ca226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:13.207901 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ce708-c2da-43ac-a0b9-1b58d70ca226 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.208185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ce708-c2da-43ac-a0b9-1b58d70ca226 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.208461 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407ce708-c2da-43ac-a0b9-1b58d70ca226 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.218688 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ce708-c2da-43ac-a0b9-1b58d70ca226 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.221048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407ce708-c2da-43ac-a0b9-1b58d70ca226 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-407ce708-c2da-43ac-a0b9-1b58d70ca226 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:13.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:13.238656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:13.239519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:13.239519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:13.239519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:13.239519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 411/819] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:13.240028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ce708-c2da-43ac-a0b9-1b58d70ca226 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:13.267442 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0293fa42-68e4-461b-820d-560c9e44c331 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:13.290723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:13.303306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd6d94-cc7f-4825-b332-3646cb736cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:13.308124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd6d94-cc7f-4825-b332-3646cb736cef None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.308124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd6d94-cc7f-4825-b332-3646cb736cef None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.308124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bd6d94-cc7f-4825-b332-3646cb736cef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.308124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd6d94-cc7f-4825-b332-3646cb736cef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.308124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bd6d94-cc7f-4825-b332-3646cb736cef None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.342043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407ce708-c2da-43ac-a0b9-1b58d70ca226 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:13.356042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:13.375295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935535b-418d-456f-bdd6-ff36c537455e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:13.376250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935535b-418d-456f-bdd6-ff36c537455e None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.376529 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935535b-418d-456f-bdd6-ff36c537455e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.376843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935535b-418d-456f-bdd6-ff36c537455e None admin] PATH_INFO: `/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/7ba12b94717a4a40b55635192840d3c3/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.378820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935535b-418d-456f-bdd6-ff36c537455e None admin] RBAC: Authorizing `identity:create_grant(project_id=71c96e5caeb44927bc21504fe54f0f98, user_id=7ba12b94717a4a40b55635192840d3c3, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.380410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935535b-418d-456f-bdd6-ff36c537455e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.398953 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 409/820] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 17:45:13 2022] POST /identity/v3/endpoints => generated 337 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:13.399733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd6d94-cc7f-4825-b332-3646cb736cef None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:13.458048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 412/821] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:45:13 2022] PUT /identity/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/7ba12b94717a4a40b55635192840d3c3/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:13.459174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935535b-418d-456f-bdd6-ff36c537455e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:13.546635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bd6d94-cc7f-4825-b332-3646cb736cef None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:13.596566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed24e81-6f9f-4c1e-8541-8cb2f3ced608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:13.597660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed24e81-6f9f-4c1e-8541-8cb2f3ced608 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.597984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed24e81-6f9f-4c1e-8541-8cb2f3ced608 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.598330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed24e81-6f9f-4c1e-8541-8cb2f3ced608 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.599495 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed24e81-6f9f-4c1e-8541-8cb2f3ced608 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:13.599780 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed24e81-6f9f-4c1e-8541-8cb2f3ced608 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.601108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed24e81-6f9f-4c1e-8541-8cb2f3ced608 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.609753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935535b-418d-456f-bdd6-ff36c537455e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:13.617182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 410/822] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:13.618478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed24e81-6f9f-4c1e-8541-8cb2f3ced608 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:13.665003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52819286-294b-4f0c-8647-ceed3ae56d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:13.665438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52819286-294b-4f0c-8647-ceed3ae56d5a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.665719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52819286-294b-4f0c-8647-ceed3ae56d5a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.665987 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52819286-294b-4f0c-8647-ceed3ae56d5a None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.666611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52819286-294b-4f0c-8647-ceed3ae56d5a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:13.666915 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52819286-294b-4f0c-8647-ceed3ae56d5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.667740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:13.670350 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52819286-294b-4f0c-8647-ceed3ae56d5a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.679402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 413/823] 10.222.0.67 () {56 vars in 1124 bytes} [Mon Sep 26 17:45:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:13.680248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52819286-294b-4f0c-8647-ceed3ae56d5a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:13.732181 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:13.741470 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed24e81-6f9f-4c1e-8541-8cb2f3ced608 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:13.777257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091abd9-b155-41cd-8d57-d9abb4318e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:13.777972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091abd9-b155-41cd-8d57-d9abb4318e95 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.778251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091abd9-b155-41cd-8d57-d9abb4318e95 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.778609 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091abd9-b155-41cd-8d57-d9abb4318e95 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.779149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091abd9-b155-41cd-8d57-d9abb4318e95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:13.779448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091abd9-b155-41cd-8d57-d9abb4318e95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.780890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091abd9-b155-41cd-8d57-d9abb4318e95 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.792812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 411/824] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:13.793566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091abd9-b155-41cd-8d57-d9abb4318e95 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:13.804204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52819286-294b-4f0c-8647-ceed3ae56d5a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:13.841798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d45a05-a0be-411d-afd9-940dbd8044e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:13.842602 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d45a05-a0be-411d-afd9-940dbd8044e2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.843265 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d45a05-a0be-411d-afd9-940dbd8044e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.843542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d45a05-a0be-411d-afd9-940dbd8044e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.852516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d45a05-a0be-411d-afd9-940dbd8044e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.853970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d45a05-a0be-411d-afd9-940dbd8044e2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.866774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f9d45a05-a0be-411d-afd9-940dbd8044e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:13.866774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:13.866774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:13.866774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:13.866774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:13.866774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:13.866774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:13.866774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:13.866774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:13.866774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:13.866774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:13.866774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:13.866774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:13.866774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:13.866774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:13.867737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:13.869990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:13.869990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:13.869990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:13.869990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:13.869990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:13.869990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:13.869990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:13.869990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:13.869990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 414/825] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:13.869990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d45a05-a0be-411d-afd9-940dbd8044e2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:13.894685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:13.901669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091abd9-b155-41cd-8d57-d9abb4318e95 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:13.942235 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84a339-96b2-4032-b8c6-1319d0a50087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:13.942757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84a339-96b2-4032-b8c6-1319d0a50087 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:13.943042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84a339-96b2-4032-b8c6-1319d0a50087 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:13.943311 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a84a339-96b2-4032-b8c6-1319d0a50087 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:13.943979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84a339-96b2-4032-b8c6-1319d0a50087 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:13.944281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84a339-96b2-4032-b8c6-1319d0a50087 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:13.945890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a84a339-96b2-4032-b8c6-1319d0a50087 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:13.955985 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 412/826] 10.222.0.67 () {56 vars in 1124 bytes} [Mon Sep 26 17:45:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:13.955985 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84a339-96b2-4032-b8c6-1319d0a50087 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:13.984702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:13.989472 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d45a05-a0be-411d-afd9-940dbd8044e2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:14.025231 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88837fd9-a813-4a20-8ec7-c3de1e2cd2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:14.026127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88837fd9-a813-4a20-8ec7-c3de1e2cd2b3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:14.026408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88837fd9-a813-4a20-8ec7-c3de1e2cd2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:14.026989 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88837fd9-a813-4a20-8ec7-c3de1e2cd2b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:14.034545 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88837fd9-a813-4a20-8ec7-c3de1e2cd2b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:14.036036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88837fd9-a813-4a20-8ec7-c3de1e2cd2b3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:14.047186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-88837fd9-a813-4a20-8ec7-c3de1e2cd2b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:14.047186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:14.047186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:14.047186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:14.047186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:14.047186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:14.047186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:14.047186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:14.047186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:14.047186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:14.047186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:14.047186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:14.047186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:14.047186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:14.047186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:14.048868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:14.049733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:14.049733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:14.049733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:14.049733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:14.049733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:14.049733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:14.049733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:14.049733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:14.053729 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 415/827] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:14.054356 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88837fd9-a813-4a20-8ec7-c3de1e2cd2b3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:14.069518 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a84a339-96b2-4032-b8c6-1319d0a50087 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:14.106046 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a38e2-2474-4634-bd28-034fa0e768c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:14.106758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a38e2-2474-4634-bd28-034fa0e768c9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:14.107032 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a38e2-2474-4634-bd28-034fa0e768c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:14.107415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a38e2-2474-4634-bd28-034fa0e768c9 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:14.108124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a38e2-2474-4634-bd28-034fa0e768c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:14.108422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a38e2-2474-4634-bd28-034fa0e768c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:14.109869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a38e2-2474-4634-bd28-034fa0e768c9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:14.127705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 413/828] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:14.127705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a38e2-2474-4634-bd28-034fa0e768c9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:14.147601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:14.165656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88837fd9-a813-4a20-8ec7-c3de1e2cd2b3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:14.203065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae0aa6-804b-46a1-ab61-e54edaa3b6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:14.204178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae0aa6-804b-46a1-ab61-e54edaa3b6d6 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:14.204466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae0aa6-804b-46a1-ab61-e54edaa3b6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:14.204769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ae0aa6-804b-46a1-ab61-e54edaa3b6d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:14.205498 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae0aa6-804b-46a1-ab61-e54edaa3b6d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:14.207277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ae0aa6-804b-46a1-ab61-e54edaa3b6d6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:14.232293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a38e2-2474-4634-bd28-034fa0e768c9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:14.262083 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 416/829] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 17:45:14 2022] POST /identity/v3/endpoints => generated 346 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:14.263283 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae0aa6-804b-46a1-ab61-e54edaa3b6d6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:14.284803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77f49a-594f-42fe-97a0-706da64c2e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:14.285913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77f49a-594f-42fe-97a0-706da64c2e67 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:14.286381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77f49a-594f-42fe-97a0-706da64c2e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:14.286823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77f49a-594f-42fe-97a0-706da64c2e67 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:14.287868 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77f49a-594f-42fe-97a0-706da64c2e67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:14.288409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77f49a-594f-42fe-97a0-706da64c2e67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:14.290069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77f49a-594f-42fe-97a0-706da64c2e67 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:14.292088 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:14.306025 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 414/830] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:14.316733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77f49a-594f-42fe-97a0-706da64c2e67 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:14.400045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ae0aa6-804b-46a1-ab61-e54edaa3b6d6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:14.410489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:14.440347 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77f49a-594f-42fe-97a0-706da64c2e67 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:14.442283 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48f55c-82e4-4a99-aa15-e1c391544cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:14.443005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48f55c-82e4-4a99-aa15-e1c391544cec None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:14.443187 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48f55c-82e4-4a99-aa15-e1c391544cec None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:14.443548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48f55c-82e4-4a99-aa15-e1c391544cec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:14.444393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48f55c-82e4-4a99-aa15-e1c391544cec None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=b9ed29a3f6be4d64bb3751f9e95eb56b) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:14.445520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48f55c-82e4-4a99-aa15-e1c391544cec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:14.446261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48f55c-82e4-4a99-aa15-e1c391544cec None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:14.459759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 417/831] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:14 2022] GET /identity/v3/role_assignments?user.id=b9ed29a3f6be4d64bb3751f9e95eb56b&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=07626899472b4e32adc0c71ce796a25d => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:14.494099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b5ae9-cf05-4474-823e-4df7952971bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:14.494786 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b5ae9-cf05-4474-823e-4df7952971bf None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:14.495065 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b5ae9-cf05-4474-823e-4df7952971bf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:14.495337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b5ae9-cf05-4474-823e-4df7952971bf None admin] PATH_INFO: `/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/1d201727ffae42c4bd704d26c03c5d9b/roles/90322db354f94f03b32ec088922baf2e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:14.512929 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b5ae9-cf05-4474-823e-4df7952971bf None admin] RBAC: Authorizing `identity:create_grant(project_id=71c96e5caeb44927bc21504fe54f0f98, user_id=1d201727ffae42c4bd704d26c03c5d9b, role_id=90322db354f94f03b32ec088922baf2e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:14.514674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b5ae9-cf05-4474-823e-4df7952971bf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:14.588623 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 415/832] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:45:14 2022] PUT /identity/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/1d201727ffae42c4bd704d26c03c5d9b/roles/90322db354f94f03b32ec088922baf2e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:15.715530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51f0ca-21e4-49af-bce8-3737b5b2c02b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:15.716082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51f0ca-21e4-49af-bce8-3737b5b2c02b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:15.716189 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51f0ca-21e4-49af-bce8-3737b5b2c02b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:15.717611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 418/833] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:15.724012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0c0a2-5618-428f-96b6-f9708e0b523e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:15.724423 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0c0a2-5618-428f-96b6-f9708e0b523e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:15.724594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0c0a2-5618-428f-96b6-f9708e0b523e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:15.742912 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-10a0c0a2-5618-428f-96b6-f9708e0b523e None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:15.744111 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-10a0c0a2-5618-428f-96b6-f9708e0b523e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:15.814433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 416/834] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:16.094612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51f0ca-21e4-49af-bce8-3737b5b2c02b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:16.177454 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:16.204941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51f0ca-21e4-49af-bce8-3737b5b2c02b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:16.241077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285dd475-9cca-4f6d-91f2-feeb466f6fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:16.241630 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dd475-9cca-4f6d-91f2-feeb466f6fbd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.241906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dd475-9cca-4f6d-91f2-feeb466f6fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.242172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285dd475-9cca-4f6d-91f2-feeb466f6fbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.252997 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285dd475-9cca-4f6d-91f2-feeb466f6fbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-285dd475-9cca-4f6d-91f2-feeb466f6fbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:16.258221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:16.259135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:16.260281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:16.260281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:16.260281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:16.260281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:16.260281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:16.260281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:16.260281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:16.260281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:16.260281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:16.260281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:16.260281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:16.260281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:16.260281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 419/835] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:16.267490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0c0a2-5618-428f-96b6-f9708e0b523e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:16.333616 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:16.379603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0c0a2-5618-428f-96b6-f9708e0b523e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:16.430470 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d4337-ac63-4b3c-b423-f1cdadba5d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:16.431141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d4337-ac63-4b3c-b423-f1cdadba5d86 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.431562 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d4337-ac63-4b3c-b423-f1cdadba5d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.431860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d4337-ac63-4b3c-b423-f1cdadba5d86 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.432530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d4337-ac63-4b3c-b423-f1cdadba5d86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:16.432830 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d4337-ac63-4b3c-b423-f1cdadba5d86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:16.434243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d4337-ac63-4b3c-b423-f1cdadba5d86 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:16.445747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 417/836] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:16.463708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285dd475-9cca-4f6d-91f2-feeb466f6fbd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:16.565581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285dd475-9cca-4f6d-91f2-feeb466f6fbd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:16.610335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62818d7b-f707-463d-99f4-e8681d8c341c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:16.611139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62818d7b-f707-463d-99f4-e8681d8c341c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.611385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62818d7b-f707-463d-99f4-e8681d8c341c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.611672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62818d7b-f707-463d-99f4-e8681d8c341c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.631693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62818d7b-f707-463d-99f4-e8681d8c341c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:16.631693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62818d7b-f707-463d-99f4-e8681d8c341c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-62818d7b-f707-463d-99f4-e8681d8c341c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:16.646909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:16.648173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:16.649039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:16.649039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:16.649039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:16.649039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:16.649039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 420/837] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:16.659689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d4337-ac63-4b3c-b423-f1cdadba5d86 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:16.757137 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f859b-503d-4c75-af65-8a7153756bff None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.758118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f859b-503d-4c75-af65-8a7153756bff None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.758616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f859b-503d-4c75-af65-8a7153756bff None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.765006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 421/838] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:16 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:16.768255 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e09bdf-01a4-4c25-82b9-ba87a6aaf166 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.768425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d4337-ac63-4b3c-b423-f1cdadba5d86 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:16.768860 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e09bdf-01a4-4c25-82b9-ba87a6aaf166 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.768860 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e09bdf-01a4-4c25-82b9-ba87a6aaf166 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.770407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 422/839] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:16.771608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb16bbe-7fa6-49d6-a140-a87124c0f352 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.771906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb16bbe-7fa6-49d6-a140-a87124c0f352 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.772171 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb16bbe-7fa6-49d6-a140-a87124c0f352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.787062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:16.802075 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4fb16bbe-7fa6-49d6-a140-a87124c0f352 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:16.803377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4fb16bbe-7fa6-49d6-a140-a87124c0f352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:16.807507 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51be9f6-19d1-42c0-ac40-1f3d427c5909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:16.808152 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51be9f6-19d1-42c0-ac40-1f3d427c5909 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.808443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51be9f6-19d1-42c0-ac40-1f3d427c5909 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.808752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51be9f6-19d1-42c0-ac40-1f3d427c5909 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.809342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51be9f6-19d1-42c0-ac40-1f3d427c5909 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:16.809642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51be9f6-19d1-42c0-ac40-1f3d427c5909 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:16.810999 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51be9f6-19d1-42c0-ac40-1f3d427c5909 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:16.826176 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 418/840] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:16.827625 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a6c3c-fb32-473e-8b85-e2d68a4eaceb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.828098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a6c3c-fb32-473e-8b85-e2d68a4eaceb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.828553 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a6c3c-fb32-473e-8b85-e2d68a4eaceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.848166 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-085a6c3c-fb32-473e-8b85-e2d68a4eaceb None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:16.849748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-085a6c3c-fb32-473e-8b85-e2d68a4eaceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:16.904042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 423/841] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:16.905029 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb16bbe-7fa6-49d6-a140-a87124c0f352 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:16.935686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 419/842] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:16.935686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945716a7-f3a2-4fc9-862b-170f4b1dcb2f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.935686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945716a7-f3a2-4fc9-862b-170f4b1dcb2f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.935686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945716a7-f3a2-4fc9-862b-170f4b1dcb2f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.935686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 420/843] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:16.935686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622fd73-18f5-4c6d-af59-9c93415bbd55 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:16.935686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622fd73-18f5-4c6d-af59-9c93415bbd55 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:16.936233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622fd73-18f5-4c6d-af59-9c93415bbd55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:16.959170 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9622fd73-18f5-4c6d-af59-9c93415bbd55 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:16.960597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9622fd73-18f5-4c6d-af59-9c93415bbd55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:17.059616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb16bbe-7fa6-49d6-a140-a87124c0f352 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:17.082884 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 421/844] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:16 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:17.086132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:17.102156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f64831-fcd5-43d2-9d63-24fc53f3e123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:17.108973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f64831-fcd5-43d2-9d63-24fc53f3e123 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:17.108973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f64831-fcd5-43d2-9d63-24fc53f3e123 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:17.108973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f64831-fcd5-43d2-9d63-24fc53f3e123 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:17.112317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f64831-fcd5-43d2-9d63-24fc53f3e123 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:17.113921 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f64831-fcd5-43d2-9d63-24fc53f3e123 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-04f64831-fcd5-43d2-9d63-24fc53f3e123 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:17.126614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:17.127769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:17.128984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:17.128984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:17.128984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:17.128984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:17.128984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:17.128984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:17.128984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:17.128984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:17.128984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 26 17:45:17.128984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:17.131126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 424/845] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:17.136430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622fd73-18f5-4c6d-af59-9c93415bbd55 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:17.172802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:17.199229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f64831-fcd5-43d2-9d63-24fc53f3e123 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:17.246915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622fd73-18f5-4c6d-af59-9c93415bbd55 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:17.284413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0b343-8875-49b5-b4c3-161980c9a33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:17.285343 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0b343-8875-49b5-b4c3-161980c9a33c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:17.285883 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0b343-8875-49b5-b4c3-161980c9a33c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:17.286396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0b343-8875-49b5-b4c3-161980c9a33c None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:17.287474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0b343-8875-49b5-b4c3-161980c9a33c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:17.287997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0b343-8875-49b5-b4c3-161980c9a33c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:17.289747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0b343-8875-49b5-b4c3-161980c9a33c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:17.306609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f64831-fcd5-43d2-9d63-24fc53f3e123 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:17.308926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 422/846] 10.222.0.67 () {56 vars in 1135 bytes} [Mon Sep 26 17:45:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:17.309917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0b343-8875-49b5-b4c3-161980c9a33c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:17.345613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665d8c7-8e86-409b-abe3-8eccb88adcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:17.346166 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665d8c7-8e86-409b-abe3-8eccb88adcbd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:17.346448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665d8c7-8e86-409b-abe3-8eccb88adcbd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:17.346725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665d8c7-8e86-409b-abe3-8eccb88adcbd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:17.347306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665d8c7-8e86-409b-abe3-8eccb88adcbd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:17.348707 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665d8c7-8e86-409b-abe3-8eccb88adcbd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4665d8c7-8e86-409b-abe3-8eccb88adcbd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:17.357676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:17.358829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:17.360238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:17.360238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:17.360238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:17.360238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:17.360238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:17.360238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:17.360238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:17.360238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:17.360238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:17.360238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:17.360238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:17.360238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:17.360238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:45:17.360238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:17.360238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 425/847] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:17.362438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665d8c7-8e86-409b-abe3-8eccb88adcbd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:17.407693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:17.413501 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:17.419942 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0b343-8875-49b5-b4c3-161980c9a33c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:17.454419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa8dc6-0a03-4934-916e-4fff43cd6e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:17.455053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa8dc6-0a03-4934-916e-4fff43cd6e6d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:17.455333 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa8dc6-0a03-4934-916e-4fff43cd6e6d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:17.455606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aa8dc6-0a03-4934-916e-4fff43cd6e6d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:17.456218 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa8dc6-0a03-4934-916e-4fff43cd6e6d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:17.457626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aa8dc6-0a03-4934-916e-4fff43cd6e6d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:17.462169 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665d8c7-8e86-409b-abe3-8eccb88adcbd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-50aa8dc6-0a03-4934-916e-4fff43cd6e6d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:17.466489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:17.467561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:17.472872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:17.472872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:17.472872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:17.472872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:17.472872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:17.472872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:17.472872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:17.472872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:17.472872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:17.472872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:17.472872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:17.472872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:17.472872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:45:17.472872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:17.472872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 423/848] 10.222.0.67 () {56 vars in 1132 bytes} [Mon Sep 26 17:45:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:17.473807 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa8dc6-0a03-4934-916e-4fff43cd6e6d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:17.501622 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e1bab-08ae-4fbc-ba02-72fb72c1ac43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:17.502229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e1bab-08ae-4fbc-ba02-72fb72c1ac43 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:17.502512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e1bab-08ae-4fbc-ba02-72fb72c1ac43 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:17.502782 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6e1bab-08ae-4fbc-ba02-72fb72c1ac43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:17.568836 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aa8dc6-0a03-4934-916e-4fff43cd6e6d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:17.569108 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e1bab-08ae-4fbc-ba02-72fb72c1ac43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:17.570552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6e1bab-08ae-4fbc-ba02-72fb72c1ac43 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:17.602173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e8ad6-b87b-483b-8ae6-3512a92ef543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:17.602726 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e8ad6-b87b-483b-8ae6-3512a92ef543 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:17.603005 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e8ad6-b87b-483b-8ae6-3512a92ef543 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:17.603277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e8ad6-b87b-483b-8ae6-3512a92ef543 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:17.613971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e8ad6-b87b-483b-8ae6-3512a92ef543 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:17.615915 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-960e8ad6-b87b-483b-8ae6-3512a92ef543 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:17.619186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:17.620287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:17.621260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:17.621260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:17.621260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:17.621260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:17.621260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:17.621260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:17.621260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:17.621260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:17.621260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:17.621260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:17.621260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:45:17.621260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:17.621260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 424/849] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:45:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:17.622062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e8ad6-b87b-483b-8ae6-3512a92ef543 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:17.640839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:17.679025 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 426/850] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:45:17.679895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e1bab-08ae-4fbc-ba02-72fb72c1ac43 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:17.725321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e8ad6-b87b-483b-8ae6-3512a92ef543 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:17.758945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599f2e7-1ba0-4607-8271-0eae70967105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:17.759538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599f2e7-1ba0-4607-8271-0eae70967105 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:17.759843 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599f2e7-1ba0-4607-8271-0eae70967105 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:17.760115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5599f2e7-1ba0-4607-8271-0eae70967105 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:17.760724 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599f2e7-1ba0-4607-8271-0eae70967105 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:17.761029 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599f2e7-1ba0-4607-8271-0eae70967105 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:17.762384 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5599f2e7-1ba0-4607-8271-0eae70967105 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:17.767687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 425/851] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:17.768456 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599f2e7-1ba0-4607-8271-0eae70967105 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:17.781321 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6e1bab-08ae-4fbc-ba02-72fb72c1ac43 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:17.814832 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d65751-27f1-47f1-a99e-b6c4f868cb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:17.815428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d65751-27f1-47f1-a99e-b6c4f868cb4f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:17.815739 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d65751-27f1-47f1-a99e-b6c4f868cb4f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:17.816008 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d65751-27f1-47f1-a99e-b6c4f868cb4f None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:17.816615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d65751-27f1-47f1-a99e-b6c4f868cb4f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:17.816914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d65751-27f1-47f1-a99e-b6c4f868cb4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:17.818251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d65751-27f1-47f1-a99e-b6c4f868cb4f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:17.822502 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 427/852] 10.222.0.67 () {56 vars in 1126 bytes} [Mon Sep 26 17:45:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:17.823262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d65751-27f1-47f1-a99e-b6c4f868cb4f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:17.834075 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:17.852655 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:17.870115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5599f2e7-1ba0-4607-8271-0eae70967105 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:17.906032 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb55e3-b14b-4df6-b95c-72deb47a402c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:17.907257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb55e3-b14b-4df6-b95c-72deb47a402c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:17.907665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb55e3-b14b-4df6-b95c-72deb47a402c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:17.907938 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb55e3-b14b-4df6-b95c-72deb47a402c None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:17.908588 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb55e3-b14b-4df6-b95c-72deb47a402c None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:17.908889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb55e3-b14b-4df6-b95c-72deb47a402c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:17.910240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cb55e3-b14b-4df6-b95c-72deb47a402c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:17.917994 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 426/853] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:45:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:17.918998 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb55e3-b14b-4df6-b95c-72deb47a402c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:17.924009 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d65751-27f1-47f1-a99e-b6c4f868cb4f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:17.958627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47ae02-4575-4f04-8f0c-627bed7db8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:17.959249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47ae02-4575-4f04-8f0c-627bed7db8fb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:17.959556 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47ae02-4575-4f04-8f0c-627bed7db8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:17.959853 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47ae02-4575-4f04-8f0c-627bed7db8fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:17.969682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47ae02-4575-4f04-8f0c-627bed7db8fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:17.971114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f47ae02-4575-4f04-8f0c-627bed7db8fb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8f47ae02-4575-4f04-8f0c-627bed7db8fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:17.986411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:17.987523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:17.988632 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:17.988632 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:17.988632 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:17.988632 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 428/854] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:17.988632 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47ae02-4575-4f04-8f0c-627bed7db8fb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.025089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb55e3-b14b-4df6-b95c-72deb47a402c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.049549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:18.058828 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7f196-feab-4a89-bd6b-f59a1e6b3d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.059453 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7f196-feab-4a89-bd6b-f59a1e6b3d2d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.059754 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7f196-feab-4a89-bd6b-f59a1e6b3d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.060018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e7f196-feab-4a89-bd6b-f59a1e6b3d2d None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.060662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7f196-feab-4a89-bd6b-f59a1e6b3d2d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:18.060962 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7f196-feab-4a89-bd6b-f59a1e6b3d2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:18.062303 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e7f196-feab-4a89-bd6b-f59a1e6b3d2d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:18.067510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 427/855] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:17 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:18.068342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7f196-feab-4a89-bd6b-f59a1e6b3d2d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.091172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:18.092213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47ae02-4575-4f04-8f0c-627bed7db8fb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.132651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e876c-9316-4137-a815-62be080d96e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.133279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e876c-9316-4137-a815-62be080d96e4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.133563 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e876c-9316-4137-a815-62be080d96e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.133834 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e876c-9316-4137-a815-62be080d96e4 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.134465 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e876c-9316-4137-a815-62be080d96e4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:18.134769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e876c-9316-4137-a815-62be080d96e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:18.136232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e876c-9316-4137-a815-62be080d96e4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:18.142861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 429/856] 10.222.0.67 () {56 vars in 1126 bytes} [Mon Sep 26 17:45:17 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:18.143745 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e876c-9316-4137-a815-62be080d96e4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.227922 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e7f196-feab-4a89-bd6b-f59a1e6b3d2d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.273924 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dfad31-ec34-4823-9872-cf6fb51a599d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.274668 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfad31-ec34-4823-9872-cf6fb51a599d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.275546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e876c-9316-4137-a815-62be080d96e4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.275692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfad31-ec34-4823-9872-cf6fb51a599d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.275692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dfad31-ec34-4823-9872-cf6fb51a599d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.292035 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dfad31-ec34-4823-9872-cf6fb51a599d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:18.293810 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dfad31-ec34-4823-9872-cf6fb51a599d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:18.311330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d0dfad31-ec34-4823-9872-cf6fb51a599d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:18.315169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:18.316357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:18.317430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:18.317430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:18.317430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:18.319346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1b8b7-972d-46c7-9974-58816b46d565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.319842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 428/857] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:18.320146 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1b8b7-972d-46c7-9974-58816b46d565 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.320488 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1b8b7-972d-46c7-9974-58816b46d565 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.320856 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1b8b7-972d-46c7-9974-58816b46d565 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.321508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1b8b7-972d-46c7-9974-58816b46d565 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:18.321816 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1b8b7-972d-46c7-9974-58816b46d565 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:18.323171 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d1b8b7-972d-46c7-9974-58816b46d565 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:18.329638 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dfad31-ec34-4823-9872-cf6fb51a599d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.343361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 430/858] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:18.349246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1b8b7-972d-46c7-9974-58816b46d565 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.410551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:18.458871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dfad31-ec34-4823-9872-cf6fb51a599d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.471887 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d1b8b7-972d-46c7-9974-58816b46d565 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.498448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07846726-a4df-45b1-8885-2801b5b06db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.499056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07846726-a4df-45b1-8885-2801b5b06db0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.499332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07846726-a4df-45b1-8885-2801b5b06db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.499715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07846726-a4df-45b1-8885-2801b5b06db0 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.500541 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07846726-a4df-45b1-8885-2801b5b06db0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:18.500848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07846726-a4df-45b1-8885-2801b5b06db0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:18.502376 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07846726-a4df-45b1-8885-2801b5b06db0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:18.508565 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79323149-fe7b-4663-83a9-34baf43a54a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.509208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79323149-fe7b-4663-83a9-34baf43a54a1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.509484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79323149-fe7b-4663-83a9-34baf43a54a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.509817 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79323149-fe7b-4663-83a9-34baf43a54a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.517398 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79323149-fe7b-4663-83a9-34baf43a54a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:18.518843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79323149-fe7b-4663-83a9-34baf43a54a1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:18.520216 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 429/859] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:18.521340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07846726-a4df-45b1-8885-2801b5b06db0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.530775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-79323149-fe7b-4663-83a9-34baf43a54a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:18.530775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:18.530775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:18.530775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:18.530775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:18.530775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:18.530775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:18.530775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:18.530775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:18.530775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:18.530775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:18.530775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:18.530775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:18.530775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:18.530775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:18.532553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:18.533423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:18.533423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:18.533423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:18.533423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:18.533423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:18.533423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:18.533423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:18.533423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:18.533423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 431/860] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:18.540593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79323149-fe7b-4663-83a9-34baf43a54a1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.601907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:18.636903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07846726-a4df-45b1-8885-2801b5b06db0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.661291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:18.667167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79323149-fe7b-4663-83a9-34baf43a54a1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.676910 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd270f-3868-4e1d-8548-22f51826d232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.677634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd270f-3868-4e1d-8548-22f51826d232 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.677988 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd270f-3868-4e1d-8548-22f51826d232 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.678334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd270f-3868-4e1d-8548-22f51826d232 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.686495 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd270f-3868-4e1d-8548-22f51826d232 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:18.688308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fd270f-3868-4e1d-8548-22f51826d232 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b6fd270f-3868-4e1d-8548-22f51826d232 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:18.704094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:18.705034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:18.705949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:18.705949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:18.705949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:18.705949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:18.705949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:18.705949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:18.705949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:18.705949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:18.705949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:45:18.705949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:18.710191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 430/861] 10.222.0.67 () {56 vars in 1117 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:18.711159 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd270f-3868-4e1d-8548-22f51826d232 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.717787 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d40ac5-f978-4c42-a0c1-2968ff787099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.718656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d40ac5-f978-4c42-a0c1-2968ff787099 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.719198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d40ac5-f978-4c42-a0c1-2968ff787099 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.719769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d40ac5-f978-4c42-a0c1-2968ff787099 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.720754 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d40ac5-f978-4c42-a0c1-2968ff787099 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:18.721330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d40ac5-f978-4c42-a0c1-2968ff787099 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:18.723074 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d40ac5-f978-4c42-a0c1-2968ff787099 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:18.738321 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 432/862] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:18.744341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d40ac5-f978-4c42-a0c1-2968ff787099 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.838390 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fd270f-3868-4e1d-8548-22f51826d232 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.862593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d40ac5-f978-4c42-a0c1-2968ff787099 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:18.866840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:18.885591 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b26ab-9476-45de-afed-24877d51466b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.889013 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b26ab-9476-45de-afed-24877d51466b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.889013 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b26ab-9476-45de-afed-24877d51466b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.889013 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2b26ab-9476-45de-afed-24877d51466b None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.889013 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b26ab-9476-45de-afed-24877d51466b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:18.889013 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b26ab-9476-45de-afed-24877d51466b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:18.891594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2b26ab-9476-45de-afed-24877d51466b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:18.916926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 431/863] 10.222.0.67 () {56 vars in 1149 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:18.920248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f12259-46d2-4cef-8004-553b81a54fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:18.921116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f12259-46d2-4cef-8004-553b81a54fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:18.921623 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f12259-46d2-4cef-8004-553b81a54fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:18.922101 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f12259-46d2-4cef-8004-553b81a54fd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:18.923243 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f12259-46d2-4cef-8004-553b81a54fd2 None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=7ba12b94717a4a40b55635192840d3c3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:18.923840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f12259-46d2-4cef-8004-553b81a54fd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:18.925270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f12259-46d2-4cef-8004-553b81a54fd2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:18.928150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b26ab-9476-45de-afed-24877d51466b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:18.932791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 433/864] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/role_assignments?user.id=7ba12b94717a4a40b55635192840d3c3&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=07626899472b4e32adc0c71ce796a25d => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:18.977124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:19.051612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2b26ab-9476-45de-afed-24877d51466b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:19.089490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee1519-48d8-43b6-828d-a7eb700d175b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:19.090122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee1519-48d8-43b6-828d-a7eb700d175b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:19.090398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee1519-48d8-43b6-828d-a7eb700d175b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:19.090666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee1519-48d8-43b6-828d-a7eb700d175b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:19.174358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee1519-48d8-43b6-828d-a7eb700d175b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:19.176178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee1519-48d8-43b6-828d-a7eb700d175b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:19.218114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:19.283630 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 432/865] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:18 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:45:19.289513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f12259-46d2-4cef-8004-553b81a54fd2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:19.397664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f12259-46d2-4cef-8004-553b81a54fd2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:19.434775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc12b66-3eb6-48f1-a379-65e7771dd098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:19.435837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc12b66-3eb6-48f1-a379-65e7771dd098 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:19.436369 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc12b66-3eb6-48f1-a379-65e7771dd098 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:19.436883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc12b66-3eb6-48f1-a379-65e7771dd098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:19.448330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc12b66-3eb6-48f1-a379-65e7771dd098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:19.450076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc12b66-3eb6-48f1-a379-65e7771dd098 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8bc12b66-3eb6-48f1-a379-65e7771dd098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:19.466820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:19.467841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:19.468732 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:19.468732 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:19.468732 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:19.471819 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 434/866] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:19.477662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee1519-48d8-43b6-828d-a7eb700d175b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:19.584745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee1519-48d8-43b6-828d-a7eb700d175b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:19.620314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bf5da-ba9c-47cf-a239-a2320166436b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:19.621249 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bf5da-ba9c-47cf-a239-a2320166436b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:19.621756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bf5da-ba9c-47cf-a239-a2320166436b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:19.622266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bf5da-ba9c-47cf-a239-a2320166436b None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:19.623097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bf5da-ba9c-47cf-a239-a2320166436b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:19.623600 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bf5da-ba9c-47cf-a239-a2320166436b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:19.625352 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bf5da-ba9c-47cf-a239-a2320166436b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:19.633027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:19.638345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 433/867] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:19.643784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc12b66-3eb6-48f1-a379-65e7771dd098 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:19.665874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:19.759638 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc12b66-3eb6-48f1-a379-65e7771dd098 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:19.795705 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06adffe-4964-4edd-9f42-1c119a00db6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:19.796345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06adffe-4964-4edd-9f42-1c119a00db6e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:19.796624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06adffe-4964-4edd-9f42-1c119a00db6e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:19.796971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06adffe-4964-4edd-9f42-1c119a00db6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:19.803943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06adffe-4964-4edd-9f42-1c119a00db6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:19.805380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f06adffe-4964-4edd-9f42-1c119a00db6e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:19.817385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f06adffe-4964-4edd-9f42-1c119a00db6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:19.817385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:19.817385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:19.817385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:19.817385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:19.817385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:19.817385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:19.817385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:19.817385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:19.817385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:19.817385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:19.817385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:19.817385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:19.817385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:19.817385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:19.818247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:19.819547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:19.819547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:19.819547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:19.819547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:19.819547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:19.819547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:19.819547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:19.819547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:19.819547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 435/868] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:19.824231 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bf5da-ba9c-47cf-a239-a2320166436b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:19.947324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bf5da-ba9c-47cf-a239-a2320166436b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:19.987014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce443e-4f1a-4b98-baeb-1b1efc5ac512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:19.987625 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce443e-4f1a-4b98-baeb-1b1efc5ac512 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:19.987963 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce443e-4f1a-4b98-baeb-1b1efc5ac512 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:19.988237 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce443e-4f1a-4b98-baeb-1b1efc5ac512 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:19.988926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce443e-4f1a-4b98-baeb-1b1efc5ac512 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:19.989227 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce443e-4f1a-4b98-baeb-1b1efc5ac512 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:19.990656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce443e-4f1a-4b98-baeb-1b1efc5ac512 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:20.004551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 434/869] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:20.010461 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06adffe-4964-4edd-9f42-1c119a00db6e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:20.096264 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:20.116039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06adffe-4964-4edd-9f42-1c119a00db6e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:20.152163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2daa573-122d-4bf9-9f79-5cdef27312c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:20.152772 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2daa573-122d-4bf9-9f79-5cdef27312c0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:20.153052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2daa573-122d-4bf9-9f79-5cdef27312c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:20.153327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2daa573-122d-4bf9-9f79-5cdef27312c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:20.154050 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2daa573-122d-4bf9-9f79-5cdef27312c0 None admin] RBAC: Adding query filter params (role.id=90322db354f94f03b32ec088922baf2e, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=1d201727ffae42c4bd704d26c03c5d9b) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:20.154360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2daa573-122d-4bf9-9f79-5cdef27312c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:20.155805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2daa573-122d-4bf9-9f79-5cdef27312c0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:20.162764 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 436/870] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:20 2022] GET /identity/v3/role_assignments?user.id=1d201727ffae42c4bd704d26c03c5d9b&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=90322db354f94f03b32ec088922baf2e => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:20.410460 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32599-c68e-4c66-8977-31a42063eaa9 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:20.410740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32599-c68e-4c66-8977-31a42063eaa9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:20.411011 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a32599-c68e-4c66-8977-31a42063eaa9 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:20.412690 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 435/871] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:20.417840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99815a45-7c03-49c4-8fab-6760c091fd87 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:20.418126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99815a45-7c03-49c4-8fab-6760c091fd87 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:20.418403 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99815a45-7c03-49c4-8fab-6760c091fd87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:20.437188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-99815a45-7c03-49c4-8fab-6760c091fd87 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:20.438939 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-99815a45-7c03-49c4-8fab-6760c091fd87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:20.503236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce188780-2d1d-4bd5-99ae-9d824cd5d74e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:20.503881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce188780-2d1d-4bd5-99ae-9d824cd5d74e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:20.504358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce188780-2d1d-4bd5-99ae-9d824cd5d74e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:20.507152 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 436/872] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:20.511105 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 437/873] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:20.511583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e729c9-3032-41b3-8f3d-5bb008da02d8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:20.512198 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e729c9-3032-41b3-8f3d-5bb008da02d8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:20.512891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e729c9-3032-41b3-8f3d-5bb008da02d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:20.532211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-15e729c9-3032-41b3-8f3d-5bb008da02d8 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:20.533873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-15e729c9-3032-41b3-8f3d-5bb008da02d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:20.589462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:20.611763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 437/874] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:20.790389 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99815a45-7c03-49c4-8fab-6760c091fd87 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:20.875939 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:20.897474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e729c9-3032-41b3-8f3d-5bb008da02d8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:20.902829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99815a45-7c03-49c4-8fab-6760c091fd87 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:20.943786 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e44135-7963-4294-82e4-9c70ca2cb240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:20.944402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e44135-7963-4294-82e4-9c70ca2cb240 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:20.944690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e44135-7963-4294-82e4-9c70ca2cb240 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:20.944962 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e44135-7963-4294-82e4-9c70ca2cb240 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:20.945625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e44135-7963-4294-82e4-9c70ca2cb240 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:20.946989 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e44135-7963-4294-82e4-9c70ca2cb240 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:21.023770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 438/875] 10.222.0.67 () {60 vars in 1141 bytes} [Mon Sep 26 17:45:20 2022] POST /identity/v3/services => generated 251 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:21.031475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e729c9-3032-41b3-8f3d-5bb008da02d8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:21.083800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077efa1c-049f-4761-8085-751250fa5dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:21.084350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077efa1c-049f-4761-8085-751250fa5dc4 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:21.084631 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077efa1c-049f-4761-8085-751250fa5dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:21.084902 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077efa1c-049f-4761-8085-751250fa5dc4 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:21.085579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077efa1c-049f-4761-8085-751250fa5dc4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:21.086987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077efa1c-049f-4761-8085-751250fa5dc4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:21.207218 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 438/876] 10.222.0.67 () {60 vars in 1141 bytes} [Mon Sep 26 17:45:20 2022] POST /identity/v3/services => generated 269 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:45:21.328053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e88034-68fb-43c4-946d-21179cd2227a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:21.328984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e88034-68fb-43c4-946d-21179cd2227a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:21.329306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e88034-68fb-43c4-946d-21179cd2227a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:21.331205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 439/877] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:21.337236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ace9ab-04f2-4f46-bf79-66e285ca6d71 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:21.337518 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ace9ab-04f2-4f46-bf79-66e285ca6d71 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:21.337716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ace9ab-04f2-4f46-bf79-66e285ca6d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:21.358114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-26ace9ab-04f2-4f46-bf79-66e285ca6d71 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:21.359322 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-26ace9ab-04f2-4f46-bf79-66e285ca6d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:21.410207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:21.449176 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 439/878] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:21.727990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e88034-68fb-43c4-946d-21179cd2227a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:21.843137 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e88034-68fb-43c4-946d-21179cd2227a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:21.881669 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636ee9d-2082-4dff-9137-ba1ab275dd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:21.882307 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636ee9d-2082-4dff-9137-ba1ab275dd08 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:21.882585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636ee9d-2082-4dff-9137-ba1ab275dd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:21.882853 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636ee9d-2082-4dff-9137-ba1ab275dd08 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:21.883433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636ee9d-2082-4dff-9137-ba1ab275dd08 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:21.884815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636ee9d-2082-4dff-9137-ba1ab275dd08 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:21.886613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5636ee9d-2082-4dff-9137-ba1ab275dd08 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:21.894870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:21.895924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:21.896966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:21.896966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:21.896966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:21.896966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:21.896966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:21.896966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:21.896966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:21.896966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:21.896966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:21.896966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:21.896966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:21.896966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:21.896966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:45:21.896966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:21.896966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 440/879] 10.222.0.67 () {56 vars in 1126 bytes} [Mon Sep 26 17:45:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:21.902349 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ace9ab-04f2-4f46-bf79-66e285ca6d71 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:22.026994 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ace9ab-04f2-4f46-bf79-66e285ca6d71 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:22.066815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560c1653-b697-413e-91b4-a992aa951ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:22.067457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c1653-b697-413e-91b4-a992aa951ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:22.067724 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c1653-b697-413e-91b4-a992aa951ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:22.067980 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560c1653-b697-413e-91b4-a992aa951ff7 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:22.068790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c1653-b697-413e-91b4-a992aa951ff7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:22.069151 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c1653-b697-413e-91b4-a992aa951ff7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:22.070861 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560c1653-b697-413e-91b4-a992aa951ff7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:22.076247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 440/880] 10.222.0.67 () {56 vars in 1122 bytes} [Mon Sep 26 17:45:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:22.085053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636ee9d-2082-4dff-9137-ba1ab275dd08 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:22.199222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636ee9d-2082-4dff-9137-ba1ab275dd08 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:22.237505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0595b7-8716-4306-91b4-237a500f1d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:22.238154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0595b7-8716-4306-91b4-237a500f1d0d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:22.238482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0595b7-8716-4306-91b4-237a500f1d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:22.239101 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0595b7-8716-4306-91b4-237a500f1d0d None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:22.239896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0595b7-8716-4306-91b4-237a500f1d0d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:22.240214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0595b7-8716-4306-91b4-237a500f1d0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:22.241568 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0595b7-8716-4306-91b4-237a500f1d0d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:22.247597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 441/881] 10.222.0.67 () {56 vars in 1122 bytes} [Mon Sep 26 17:45:22 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:22.462083 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98133c-1fec-46c0-b08b-07895f107246 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:22.462432 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98133c-1fec-46c0-b08b-07895f107246 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:22.462619 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c98133c-1fec-46c0-b08b-07895f107246 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:22.464241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 441/882] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:22.469912 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb86f1-b50f-4fd1-a354-70875cccaa23 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:22.470269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb86f1-b50f-4fd1-a354-70875cccaa23 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:22.470538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cb86f1-b50f-4fd1-a354-70875cccaa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:22.489532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-59cb86f1-b50f-4fd1-a354-70875cccaa23 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:22.490651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-59cb86f1-b50f-4fd1-a354-70875cccaa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:22.550639 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:22.564980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 442/883] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:22.837658 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98133c-1fec-46c0-b08b-07895f107246 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:22.873574 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:22.944452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c98133c-1fec-46c0-b08b-07895f107246 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:22.997116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4e1d9-80e3-40e6-b755-8eceed12c975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:22.997749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4e1d9-80e3-40e6-b755-8eceed12c975 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:22.998024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4e1d9-80e3-40e6-b755-8eceed12c975 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:22.998293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4e1d9-80e3-40e6-b755-8eceed12c975 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:23.009604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4e1d9-80e3-40e6-b755-8eceed12c975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-76d4e1d9-80e3-40e6-b755-8eceed12c975 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:23.014719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:23.015633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:23.016613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:23.016613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:23.016613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:23.016613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:23.016613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:23.016613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:23.016613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:23.016613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:23.016613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:23.016613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:23.016613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:45:23.016613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:23.016613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 442/884] 10.222.0.67 () {56 vars in 1120 bytes} [Mon Sep 26 17:45:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:23.023867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb86f1-b50f-4fd1-a354-70875cccaa23 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:23.127051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cb86f1-b50f-4fd1-a354-70875cccaa23 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:23.163453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989dccb-4fc2-4d4e-8799-5dfcc4d26e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:23.164483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989dccb-4fc2-4d4e-8799-5dfcc4d26e09 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:23.164974 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989dccb-4fc2-4d4e-8799-5dfcc4d26e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:23.165489 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989dccb-4fc2-4d4e-8799-5dfcc4d26e09 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:23.166487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989dccb-4fc2-4d4e-8799-5dfcc4d26e09 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:23.167031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989dccb-4fc2-4d4e-8799-5dfcc4d26e09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:23.168636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989dccb-4fc2-4d4e-8799-5dfcc4d26e09 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:23.178501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 443/885] 10.222.0.67 () {56 vars in 1115 bytes} [Mon Sep 26 17:45:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:23.184773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4e1d9-80e3-40e6-b755-8eceed12c975 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:23.294329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:23.302259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4e1d9-80e3-40e6-b755-8eceed12c975 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:23.333275 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7ae45-2614-4743-a4ee-9d8c4c0ff441 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:23.333606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7ae45-2614-4743-a4ee-9d8c4c0ff441 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:23.333822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab7ae45-2614-4743-a4ee-9d8c4c0ff441 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:23.335411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 444/886] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:23.338785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3afb8-5153-4592-8a3b-c866262330df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:23.339505 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3afb8-5153-4592-8a3b-c866262330df None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:23.339829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3afb8-5153-4592-8a3b-c866262330df None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:23.340102 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3afb8-5153-4592-8a3b-c866262330df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:23.340385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2872195-68d6-43ab-8ba5-89761349da67 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:23.340677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2872195-68d6-43ab-8ba5-89761349da67 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:23.340952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2872195-68d6-43ab-8ba5-89761349da67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:23.352939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3afb8-5153-4592-8a3b-c866262330df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:23.354383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b3afb8-5153-4592-8a3b-c866262330df None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:23.362700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a2872195-68d6-43ab-8ba5-89761349da67 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:23.363762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a2872195-68d6-43ab-8ba5-89761349da67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a0b3afb8-5153-4592-8a3b-c866262330df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:23.376889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:23.377915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:23.378801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:23.378801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:23.378801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:23.378801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 443/887] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:23.383528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3afb8-5153-4592-8a3b-c866262330df None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:23.427898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:23.455683 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 445/888] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:23.496210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3afb8-5153-4592-8a3b-c866262330df None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:23.533806 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ca832e-360e-4b08-9394-be85ac38933c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:23.534461 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca832e-360e-4b08-9394-be85ac38933c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:23.534745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca832e-360e-4b08-9394-be85ac38933c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:23.535017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca832e-360e-4b08-9394-be85ac38933c None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:23.535666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca832e-360e-4b08-9394-be85ac38933c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:23.535973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca832e-360e-4b08-9394-be85ac38933c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:23.537440 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ca832e-360e-4b08-9394-be85ac38933c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:23.552194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 444/889] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:23.557936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2872195-68d6-43ab-8ba5-89761349da67 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:23.671000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2872195-68d6-43ab-8ba5-89761349da67 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:23.678146 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fba50-cab8-49af-8cb5-8bd1361d8a6a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:23.678510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fba50-cab8-49af-8cb5-8bd1361d8a6a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:23.678856 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751fba50-cab8-49af-8cb5-8bd1361d8a6a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:23.684483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 445/890] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:23.685862 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffc560-73ef-446f-9005-b5d44abb59e3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:23.686216 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffc560-73ef-446f-9005-b5d44abb59e3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:23.686565 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffc560-73ef-446f-9005-b5d44abb59e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:23.699674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:23.706462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d1ffc560-73ef-446f-9005-b5d44abb59e3 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:23.708001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d1ffc560-73ef-446f-9005-b5d44abb59e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:23.708430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21ceff-3468-49de-b627-f7b9a5a78a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:23.709075 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21ceff-3468-49de-b627-f7b9a5a78a31 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:23.709356 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21ceff-3468-49de-b627-f7b9a5a78a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:23.709628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a21ceff-3468-49de-b627-f7b9a5a78a31 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:23.722153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21ceff-3468-49de-b627-f7b9a5a78a31 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:23.723672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a21ceff-3468-49de-b627-f7b9a5a78a31 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8a21ceff-3468-49de-b627-f7b9a5a78a31 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:23.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:23.738271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:23.739201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:23.739201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:23.739201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:23.739201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:23.739201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:23.739201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:23.739201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:23.739201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:23.739201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:45:23.739201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:23.739790 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 446/891] 10.222.0.67 () {56 vars in 1117 bytes} [Mon Sep 26 17:45:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:23.740056 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21ceff-3468-49de-b627-f7b9a5a78a31 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:23.787719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 446/892] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:23 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:23.787719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffc560-73ef-446f-9005-b5d44abb59e3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:23.796653 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:23.867936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a21ceff-3468-49de-b627-f7b9a5a78a31 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:23.903900 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2de6d-01a2-4883-91d9-dc3c02b3c42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:23.904748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2de6d-01a2-4883-91d9-dc3c02b3c42c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:23.905079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2de6d-01a2-4883-91d9-dc3c02b3c42c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:23.905447 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2de6d-01a2-4883-91d9-dc3c02b3c42c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:23.906124 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2de6d-01a2-4883-91d9-dc3c02b3c42c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:23.908594 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea2de6d-01a2-4883-91d9-dc3c02b3c42c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-9ea2de6d-01a2-4883-91d9-dc3c02b3c42c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:23.917709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:23.918897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:23.919943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:23.919943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:23.919943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:23.919943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:23.919943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:23.919943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:23.919943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:23.919943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:23.919943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:23.919943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:23.919943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:23.919943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:23.919943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:45:23.919943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:23.921463 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffc560-73ef-446f-9005-b5d44abb59e3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:23.928108 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 447/893] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:23 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:23.929555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2de6d-01a2-4883-91d9-dc3c02b3c42c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:23.966634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336e9a4-2e1b-4535-a80b-ef900ecdafd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:23.968033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336e9a4-2e1b-4535-a80b-ef900ecdafd6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:23.969551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336e9a4-2e1b-4535-a80b-ef900ecdafd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:23.970606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b336e9a4-2e1b-4535-a80b-ef900ecdafd6 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:23.971464 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336e9a4-2e1b-4535-a80b-ef900ecdafd6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:23.971891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336e9a4-2e1b-4535-a80b-ef900ecdafd6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:23.973566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b336e9a4-2e1b-4535-a80b-ef900ecdafd6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:23.982684 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:24.009585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 447/894] 10.222.0.67 () {56 vars in 1149 bytes} [Mon Sep 26 17:45:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:24.011044 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336e9a4-2e1b-4535-a80b-ef900ecdafd6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:24.060814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2de6d-01a2-4883-91d9-dc3c02b3c42c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:24.070753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:24.100102 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fb4e7-7939-4871-b97e-6dd80e01c0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:24.101144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fb4e7-7939-4871-b97e-6dd80e01c0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.101701 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fb4e7-7939-4871-b97e-6dd80e01c0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.101978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953fb4e7-7939-4871-b97e-6dd80e01c0e5 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.102605 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953fb4e7-7939-4871-b97e-6dd80e01c0e5 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:24.102910 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953fb4e7-7939-4871-b97e-6dd80e01c0e5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:24.104939 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953fb4e7-7939-4871-b97e-6dd80e01c0e5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:24.110582 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 448/895] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:23 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:24.111404 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fb4e7-7939-4871-b97e-6dd80e01c0e5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:24.129044 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b336e9a4-2e1b-4535-a80b-ef900ecdafd6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:24.164820 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fa598-78d2-41fd-87de-00bfa9192fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:24.165428 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fa598-78d2-41fd-87de-00bfa9192fbb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.165704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fa598-78d2-41fd-87de-00bfa9192fbb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.165970 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92fa598-78d2-41fd-87de-00bfa9192fbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.226689 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953fb4e7-7939-4871-b97e-6dd80e01c0e5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:24.232877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:24.242775 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92fa598-78d2-41fd-87de-00bfa9192fbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:24.244281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92fa598-78d2-41fd-87de-00bfa9192fbb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:24.261120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afe06f-c339-479a-9433-b4fca9d44523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:24.261749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afe06f-c339-479a-9433-b4fca9d44523 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.262033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afe06f-c339-479a-9433-b4fca9d44523 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.262303 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1afe06f-c339-479a-9433-b4fca9d44523 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.262876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1afe06f-c339-479a-9433-b4fca9d44523 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:24.264652 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1afe06f-c339-479a-9433-b4fca9d44523 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e1afe06f-c339-479a-9433-b4fca9d44523 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:24.274879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:24.275881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:24.277708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:24.277708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:24.277708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:24.277708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:24.277708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:24.277708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:24.277708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:24.277708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:24.277708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:24.277708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:24.277708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:24.277708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:24.277708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:45:24.277708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:24.277708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 449/896] 10.222.0.67 () {56 vars in 1132 bytes} [Mon Sep 26 17:45:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:24.278492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afe06f-c339-479a-9433-b4fca9d44523 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:24.318104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:24.358610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 448/897] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:24 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:45:24.359809 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fa598-78d2-41fd-87de-00bfa9192fbb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:24.392947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1afe06f-c339-479a-9433-b4fca9d44523 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:24.433604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4425b-7fbd-492d-8e3e-7c4a37b7e3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:24.434130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4425b-7fbd-492d-8e3e-7c4a37b7e3e4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.434408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4425b-7fbd-492d-8e3e-7c4a37b7e3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.435353 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f4425b-7fbd-492d-8e3e-7c4a37b7e3e4 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.435353 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4425b-7fbd-492d-8e3e-7c4a37b7e3e4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:24.435595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4425b-7fbd-492d-8e3e-7c4a37b7e3e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:24.436946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f4425b-7fbd-492d-8e3e-7c4a37b7e3e4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:24.447164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 450/898] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:24.447164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4425b-7fbd-492d-8e3e-7c4a37b7e3e4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:24.470446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:24.480889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92fa598-78d2-41fd-87de-00bfa9192fbb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:24.518637 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb1ddf-1bd2-49be-a713-3535e87d421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:24.519796 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb1ddf-1bd2-49be-a713-3535e87d421d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.520385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb1ddf-1bd2-49be-a713-3535e87d421d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.520730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffb1ddf-1bd2-49be-a713-3535e87d421d None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.521360 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb1ddf-1bd2-49be-a713-3535e87d421d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:24.521662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb1ddf-1bd2-49be-a713-3535e87d421d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:24.530993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffb1ddf-1bd2-49be-a713-3535e87d421d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:24.536678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 449/899] 10.222.0.67 () {56 vars in 1126 bytes} [Mon Sep 26 17:45:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:24.542314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb1ddf-1bd2-49be-a713-3535e87d421d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:24.554257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:24.557581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f4425b-7fbd-492d-8e3e-7c4a37b7e3e4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:24.594937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088053f-3f0f-47e1-881b-48000a6d00ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:24.595572 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088053f-3f0f-47e1-881b-48000a6d00ec None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.595873 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088053f-3f0f-47e1-881b-48000a6d00ec None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.596141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088053f-3f0f-47e1-881b-48000a6d00ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.606263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088053f-3f0f-47e1-881b-48000a6d00ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:24.607713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6088053f-3f0f-47e1-881b-48000a6d00ec None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6088053f-3f0f-47e1-881b-48000a6d00ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:24.623841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:24.624990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:24.626676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:24.626676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:24.626676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:24.626676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 451/900] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:24.626676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088053f-3f0f-47e1-881b-48000a6d00ec None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:24.654033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffb1ddf-1bd2-49be-a713-3535e87d421d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:24.689683 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d2df7-577c-449c-a01a-e8650d567af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:24.690291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d2df7-577c-449c-a01a-e8650d567af6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.690572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d2df7-577c-449c-a01a-e8650d567af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.690934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d2df7-577c-449c-a01a-e8650d567af6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.691528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d2df7-577c-449c-a01a-e8650d567af6 None admin] RBAC: Adding query filter params (interface=public, service_id=af7120e88f3543eb90930c8df6b2569f, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:24.691864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d2df7-577c-449c-a01a-e8650d567af6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:24.693259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d2df7-577c-449c-a01a-e8650d567af6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:24.700555 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 450/901] 10.222.0.67 () {56 vars in 1251 bytes} [Mon Sep 26 17:45:24 2022] GET /identity/v3/endpoints?service_id=af7120e88f3543eb90930c8df6b2569f&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:24.701703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc1a68-bae5-490a-a077-eea461595c8f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.702060 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc1a68-bae5-490a-a077-eea461595c8f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.702338 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efc1a68-bae5-490a-a077-eea461595c8f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.704330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 451/902] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:24.704944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc1a68-bae5-490a-a077-eea461595c8f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:24.731969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:24.746409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088053f-3f0f-47e1-881b-48000a6d00ec None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:24.779984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0fbaa-be31-4e89-bff6-c46362df57d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:24.780591 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0fbaa-be31-4e89-bff6-c46362df57d4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.780937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0fbaa-be31-4e89-bff6-c46362df57d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.781127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0fbaa-be31-4e89-bff6-c46362df57d4 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.782668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c0fbaa-be31-4e89-bff6-c46362df57d4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:24.782977 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c0fbaa-be31-4e89-bff6-c46362df57d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:24.784369 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c0fbaa-be31-4e89-bff6-c46362df57d4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:24.789163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 452/903] 10.222.0.67 () {56 vars in 1126 bytes} [Mon Sep 26 17:45:24 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:24.789942 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0fbaa-be31-4e89-bff6-c46362df57d4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:24.808584 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:24.813586 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efc1a68-bae5-490a-a077-eea461595c8f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:24.849223 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dddb2b6-5fca-4dc7-8c6c-7c21a9be1aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:24.849949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddb2b6-5fca-4dc7-8c6c-7c21a9be1aba None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.850239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddb2b6-5fca-4dc7-8c6c-7c21a9be1aba None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.850508 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dddb2b6-5fca-4dc7-8c6c-7c21a9be1aba None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.851475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddb2b6-5fca-4dc7-8c6c-7c21a9be1aba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:24.851810 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddb2b6-5fca-4dc7-8c6c-7c21a9be1aba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:24.853598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dddb2b6-5fca-4dc7-8c6c-7c21a9be1aba None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:24.866016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 452/904] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:24.867183 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c5b71-22c9-4d6d-b3eb-e1c614a8a125 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.867461 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c5b71-22c9-4d6d-b3eb-e1c614a8a125 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.867767 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c5b71-22c9-4d6d-b3eb-e1c614a8a125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.885150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1a0c5b71-22c9-4d6d-b3eb-e1c614a8a125 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:24.886329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1a0c5b71-22c9-4d6d-b3eb-e1c614a8a125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:24.894091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0fbaa-be31-4e89-bff6-c46362df57d4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:24.927720 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b810dd7-0bbd-481e-a16e-76b0e0150d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:24.928323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b810dd7-0bbd-481e-a16e-76b0e0150d2b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:24.928595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b810dd7-0bbd-481e-a16e-76b0e0150d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:24.928862 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b810dd7-0bbd-481e-a16e-76b0e0150d2b None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:24.929454 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b810dd7-0bbd-481e-a16e-76b0e0150d2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:24.930798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b810dd7-0bbd-481e-a16e-76b0e0150d2b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:24.936301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 453/905] 10.222.0.67 () {56 vars in 1087 bytes} [Mon Sep 26 17:45:24 2022] GET /identity/v3/services => generated 1268 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:45:24.937065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b810dd7-0bbd-481e-a16e-76b0e0150d2b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:24.960109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 453/906] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:24.960900 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c5b71-22c9-4d6d-b3eb-e1c614a8a125 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:24.993670 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:25.071625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:25.099573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b810dd7-0bbd-481e-a16e-76b0e0150d2b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:25.109034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c5b71-22c9-4d6d-b3eb-e1c614a8a125 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:25.145660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc90a5c-9bfc-4ea0-ac86-7ab49d6ab69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:25.146435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc90a5c-9bfc-4ea0-ac86-7ab49d6ab69c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:25.146886 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc90a5c-9bfc-4ea0-ac86-7ab49d6ab69c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:25.147271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc90a5c-9bfc-4ea0-ac86-7ab49d6ab69c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:25.148882 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc90a5c-9bfc-4ea0-ac86-7ab49d6ab69c None admin] RBAC: Adding query filter params (interface=public, service_id=e85feee7f1ce444cb2552da5c919e5a4, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:25.149290 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc90a5c-9bfc-4ea0-ac86-7ab49d6ab69c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:25.150752 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc90a5c-9bfc-4ea0-ac86-7ab49d6ab69c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:25.154281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758530d1-265f-4a8a-aaa0-820d9f8f3395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:25.155192 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758530d1-265f-4a8a-aaa0-820d9f8f3395 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:25.155472 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758530d1-265f-4a8a-aaa0-820d9f8f3395 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:25.156160 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758530d1-265f-4a8a-aaa0-820d9f8f3395 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:25.156987 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 454/907] 10.222.0.67 () {56 vars in 1251 bytes} [Mon Sep 26 17:45:24 2022] GET /identity/v3/endpoints?service_id=e85feee7f1ce444cb2552da5c919e5a4&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:25.162585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc90a5c-9bfc-4ea0-ac86-7ab49d6ab69c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:25.164180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758530d1-265f-4a8a-aaa0-820d9f8f3395 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:25.165830 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758530d1-265f-4a8a-aaa0-820d9f8f3395 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:25.180497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-758530d1-265f-4a8a-aaa0-820d9f8f3395 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:25.180497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:25.180497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:25.180497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:25.180497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:25.180497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:25.180497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:25.180497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:25.180497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:25.180497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:25.180497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:25.180497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:25.180497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:25.180497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:25.180497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:25.181398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:25.182252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:25.182252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:25.182252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:25.182252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:25.182252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:25.182252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:25.182252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:25.182252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:25.182252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 454/908] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:25.188020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758530d1-265f-4a8a-aaa0-820d9f8f3395 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:25.267316 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:25.285627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc90a5c-9bfc-4ea0-ac86-7ab49d6ab69c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:25.307132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758530d1-265f-4a8a-aaa0-820d9f8f3395 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:25.322513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef127da8-33d5-40e1-bd62-07c35ceb08c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:25.323126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef127da8-33d5-40e1-bd62-07c35ceb08c8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:25.323402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef127da8-33d5-40e1-bd62-07c35ceb08c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:25.323795 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef127da8-33d5-40e1-bd62-07c35ceb08c8 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:25.324293 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef127da8-33d5-40e1-bd62-07c35ceb08c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:25.325647 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef127da8-33d5-40e1-bd62-07c35ceb08c8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:25.339692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 455/909] 10.222.0.67 () {56 vars in 1087 bytes} [Mon Sep 26 17:45:25 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 17:45:25.339692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef127da8-33d5-40e1-bd62-07c35ceb08c8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:25.345017 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:25.363796 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d5d7b-1f25-4a8a-a558-660573762d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:25.364599 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d5d7b-1f25-4a8a-a558-660573762d46 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:25.365315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d5d7b-1f25-4a8a-a558-660573762d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:25.365597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9d5d7b-1f25-4a8a-a558-660573762d46 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:25.366841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d5d7b-1f25-4a8a-a558-660573762d46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:25.367153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d5d7b-1f25-4a8a-a558-660573762d46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:25.369411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9d5d7b-1f25-4a8a-a558-660573762d46 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:25.388015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 455/910] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:25.396048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d5d7b-1f25-4a8a-a558-660573762d46 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:25.456183 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef127da8-33d5-40e1-bd62-07c35ceb08c8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:25.501360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b06a0e-700e-4bf2-b8e9-adbd906ec5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:25.502874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b06a0e-700e-4bf2-b8e9-adbd906ec5b3 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:25.503267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b06a0e-700e-4bf2-b8e9-adbd906ec5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:25.503641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b06a0e-700e-4bf2-b8e9-adbd906ec5b3 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:25.504441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b06a0e-700e-4bf2-b8e9-adbd906ec5b3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:25.506839 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b06a0e-700e-4bf2-b8e9-adbd906ec5b3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:25.532974 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9d5d7b-1f25-4a8a-a558-660573762d46 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:25.565884 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:25.570945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea44fc-3518-4afc-b3ea-088b9ce8e63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:25.571563 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea44fc-3518-4afc-b3ea-088b9ce8e63b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:25.571875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea44fc-3518-4afc-b3ea-088b9ce8e63b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:25.572147 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea44fc-3518-4afc-b3ea-088b9ce8e63b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:25.573705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea44fc-3518-4afc-b3ea-088b9ce8e63b None admin] RBAC: Adding query filter params (role.id=fb6eaf230a474167b3a3726cbd2d35aa, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=1d201727ffae42c4bd704d26c03c5d9b) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:25.573705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea44fc-3518-4afc-b3ea-088b9ce8e63b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:25.574592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ea44fc-3518-4afc-b3ea-088b9ce8e63b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:25.583689 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 456/911] 10.222.0.67 () {60 vars in 1141 bytes} [Mon Sep 26 17:45:25 2022] POST /identity/v3/services => generated 241 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:25.583890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 456/912] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:25 2022] GET /identity/v3/role_assignments?user.id=1d201727ffae42c4bd704d26c03c5d9b&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=fb6eaf230a474167b3a3726cbd2d35aa => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:27.180306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77feeca-d78b-417f-a548-58c8cb38c282 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.181297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77feeca-d78b-417f-a548-58c8cb38c282 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.181789 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77feeca-d78b-417f-a548-58c8cb38c282 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.183529 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 457/913] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:27.189823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54f4de-2210-4ef3-90f9-143c13cfbe49 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.190674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54f4de-2210-4ef3-90f9-143c13cfbe49 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.191147 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d54f4de-2210-4ef3-90f9-143c13cfbe49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.210386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3d54f4de-2210-4ef3-90f9-143c13cfbe49 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:27.212003 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3d54f4de-2210-4ef3-90f9-143c13cfbe49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:27.293112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 457/914] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:27.564028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77feeca-d78b-417f-a548-58c8cb38c282 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:27.635495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:27.668753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f2fd0-0b54-4379-83bb-afd5350fc4f9 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.669405 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f2fd0-0b54-4379-83bb-afd5350fc4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.669777 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f2fd0-0b54-4379-83bb-afd5350fc4f9 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.671805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 458/915] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:27.673018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77feeca-d78b-417f-a548-58c8cb38c282 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:27.677155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24535794-95d7-4744-89ca-0e8664cd14c6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.677664 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24535794-95d7-4744-89ca-0e8664cd14c6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.678134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24535794-95d7-4744-89ca-0e8664cd14c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.699088 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-24535794-95d7-4744-89ca-0e8664cd14c6 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:27.700132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-24535794-95d7-4744-89ca-0e8664cd14c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:27.714887 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e08eab-de1c-45a5-983b-6687490cf1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:27.715523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e08eab-de1c-45a5-983b-6687490cf1aa None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.715838 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e08eab-de1c-45a5-983b-6687490cf1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.716165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e08eab-de1c-45a5-983b-6687490cf1aa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.716791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e08eab-de1c-45a5-983b-6687490cf1aa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:27.718292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e08eab-de1c-45a5-983b-6687490cf1aa None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a4e08eab-de1c-45a5-983b-6687490cf1aa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:27.728713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:27.729740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:27.730548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:27.730548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:27.730548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:27.730548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:27.730548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:27.730548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:27.730548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:27.730548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:27.730548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:27.730548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:27.730548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:27.730548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:27.730548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 26 17:45:27.730548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:27.730548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 458/916] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:27.738060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e08eab-de1c-45a5-983b-6687490cf1aa None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:27.795760 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 459/917] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:27.863433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad79037-1893-4e42-938e-334354eebe06 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.863623 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad79037-1893-4e42-938e-334354eebe06 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.863924 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad79037-1893-4e42-938e-334354eebe06 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.865522 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 460/918] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:27.865884 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e08eab-de1c-45a5-983b-6687490cf1aa None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:27.870314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f7c91-66cf-4b03-adab-9db86ee72b65 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.870605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f7c91-66cf-4b03-adab-9db86ee72b65 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.870883 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f7c91-66cf-4b03-adab-9db86ee72b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.890170 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8c3f7c91-66cf-4b03-adab-9db86ee72b65 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:27.892040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8c3f7c91-66cf-4b03-adab-9db86ee72b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:27.906059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aecba1-9d5c-42b7-9e13-e6882b00f964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:27.907959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aecba1-9d5c-42b7-9e13-e6882b00f964 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.908505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aecba1-9d5c-42b7-9e13-e6882b00f964 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.908971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aecba1-9d5c-42b7-9e13-e6882b00f964 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.909816 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aecba1-9d5c-42b7-9e13-e6882b00f964 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:27.910377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aecba1-9d5c-42b7-9e13-e6882b00f964 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:27.912123 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aecba1-9d5c-42b7-9e13-e6882b00f964 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:27.913549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:27.918651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 459/919] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:27.925196 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053979a9-346a-4a38-9346-f11a8fe68869 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.927433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053979a9-346a-4a38-9346-f11a8fe68869 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.927433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053979a9-346a-4a38-9346-f11a8fe68869 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.927433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 460/920] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:27.928237 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053979a9-346a-4a38-9346-f11a8fe68869 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:27.944501 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:27.966416 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 461/921] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:27.967796 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b23e2-d9d8-4c0c-a721-67ad3767c048 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:27.968078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b23e2-d9d8-4c0c-a721-67ad3767c048 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:27.968363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b23e2-d9d8-4c0c-a721-67ad3767c048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:27.998450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ab7b23e2-d9d8-4c0c-a721-67ad3767c048 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:27.999655 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ab7b23e2-d9d8-4c0c-a721-67ad3767c048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:28.070855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053979a9-346a-4a38-9346-f11a8fe68869 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.079669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 462/922] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:28.093337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b23e2-d9d8-4c0c-a721-67ad3767c048 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.116637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be543ea-b8d1-479f-ace3-b2b7771dda29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.117224 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be543ea-b8d1-479f-ace3-b2b7771dda29 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.117501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be543ea-b8d1-479f-ace3-b2b7771dda29 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.117772 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be543ea-b8d1-479f-ace3-b2b7771dda29 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.118505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be543ea-b8d1-479f-ace3-b2b7771dda29 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:28.118859 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be543ea-b8d1-479f-ace3-b2b7771dda29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:28.120251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be543ea-b8d1-479f-ace3-b2b7771dda29 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:28.126269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 461/923] 10.222.0.67 () {56 vars in 1114 bytes} [Mon Sep 26 17:45:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:28.131722 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be543ea-b8d1-479f-ace3-b2b7771dda29 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.198697 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:28.222713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b23e2-d9d8-4c0c-a721-67ad3767c048 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.242083 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:28.242757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be543ea-b8d1-479f-ace3-b2b7771dda29 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.266193 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2de757-d6c0-4b30-b59b-da2dfa343db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.267161 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2de757-d6c0-4b30-b59b-da2dfa343db0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.267449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2de757-d6c0-4b30-b59b-da2dfa343db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.267996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2de757-d6c0-4b30-b59b-da2dfa343db0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.268904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2de757-d6c0-4b30-b59b-da2dfa343db0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:28.270402 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2de757-d6c0-4b30-b59b-da2dfa343db0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6a2de757-d6c0-4b30-b59b-da2dfa343db0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:28.279357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:28.280677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:28.281964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:28.281964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:28.281964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:28.281964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:28.281964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:28.281964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:28.281964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:28.281964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:28.281964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:28.281964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:28.281964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:28.281964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:28.281964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 26 17:45:28.281964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:28.281964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 463/924] 10.222.0.67 () {56 vars in 1132 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:28.284857 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2de757-d6c0-4b30-b59b-da2dfa343db0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.288850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd8789-253f-4fa2-9c13-8e0cea249946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.289547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd8789-253f-4fa2-9c13-8e0cea249946 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.289831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd8789-253f-4fa2-9c13-8e0cea249946 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.290205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd8789-253f-4fa2-9c13-8e0cea249946 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.290866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd8789-253f-4fa2-9c13-8e0cea249946 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:28.292257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd8789-253f-4fa2-9c13-8e0cea249946 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:28.347923 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 462/925] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 17:45:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:45:28.348617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd8789-253f-4fa2-9c13-8e0cea249946 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.402287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2de757-d6c0-4b30-b59b-da2dfa343db0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.448592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b8356-5422-4cd3-b5ab-55fe6d3c6b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.449567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b8356-5422-4cd3-b5ab-55fe6d3c6b1e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.450028 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b8356-5422-4cd3-b5ab-55fe6d3c6b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.450479 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b8356-5422-4cd3-b5ab-55fe6d3c6b1e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.463805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b8356-5422-4cd3-b5ab-55fe6d3c6b1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:28.464929 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd8789-253f-4fa2-9c13-8e0cea249946 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f84b8356-5422-4cd3-b5ab-55fe6d3c6b1e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:28.470745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:28.471774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:28.472642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:28.472642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:28.472642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:28.472642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:28.472642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:28.472642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:28.472642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:28.472642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:28.472642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:28.472642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:28.472642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:45:28.472642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:28.473543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 464/926] 10.222.0.67 () {56 vars in 1120 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:28.474207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b8356-5422-4cd3-b5ab-55fe6d3c6b1e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.487360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:28.508928 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e0a68-a5c7-488e-8571-3cd8e9971c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.509988 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e0a68-a5c7-488e-8571-3cd8e9971c48 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.510466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e0a68-a5c7-488e-8571-3cd8e9971c48 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.510601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:28.510747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e0a68-a5c7-488e-8571-3cd8e9971c48 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.511488 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e0a68-a5c7-488e-8571-3cd8e9971c48 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:28.511863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e0a68-a5c7-488e-8571-3cd8e9971c48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:28.513323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68e0a68-a5c7-488e-8571-3cd8e9971c48 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:28.519718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 463/927] 10.222.0.67 () {56 vars in 1126 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:28.525128 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e0a68-a5c7-488e-8571-3cd8e9971c48 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.600408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b8356-5422-4cd3-b5ab-55fe6d3c6b1e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.635969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935209a-7fe4-4ccd-b0ae-00aa34b885ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.636592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935209a-7fe4-4ccd-b0ae-00aa34b885ee None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.636865 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935209a-7fe4-4ccd-b0ae-00aa34b885ee None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.637141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2935209a-7fe4-4ccd-b0ae-00aa34b885ee None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.637716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935209a-7fe4-4ccd-b0ae-00aa34b885ee None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:28.639372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2935209a-7fe4-4ccd-b0ae-00aa34b885ee None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:28.643189 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e0a68-a5c7-488e-8571-3cd8e9971c48 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2935209a-7fe4-4ccd-b0ae-00aa34b885ee None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:28.648927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:28.649920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:28.650737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:28.650737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:28.650737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:28.650737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:28.650737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:28.650737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:28.650737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:28.650737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:28.650737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:28.650737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:28.650737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:28.650737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:28.650737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:45:28.650737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:28.655073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 465/928] 10.222.0.67 () {56 vars in 1126 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:28.655658 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935209a-7fe4-4ccd-b0ae-00aa34b885ee None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.681674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695002d6-6b37-4a3a-b514-247111fcf604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.682618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695002d6-6b37-4a3a-b514-247111fcf604 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.682916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695002d6-6b37-4a3a-b514-247111fcf604 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.683441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695002d6-6b37-4a3a-b514-247111fcf604 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.684139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695002d6-6b37-4a3a-b514-247111fcf604 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:28.684456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695002d6-6b37-4a3a-b514-247111fcf604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:28.685831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695002d6-6b37-4a3a-b514-247111fcf604 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:28.694756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 464/929] 10.222.0.67 () {56 vars in 1115 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:28.695608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695002d6-6b37-4a3a-b514-247111fcf604 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.744766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:28.765351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:28.771753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2935209a-7fe4-4ccd-b0ae-00aa34b885ee None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.805162 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd66fd0-a2d1-4de5-a353-acaa80749dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.805773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd66fd0-a2d1-4de5-a353-acaa80749dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.806055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd66fd0-a2d1-4de5-a353-acaa80749dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.806380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd66fd0-a2d1-4de5-a353-acaa80749dd6 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.807002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd66fd0-a2d1-4de5-a353-acaa80749dd6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:28.807306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd66fd0-a2d1-4de5-a353-acaa80749dd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:28.808822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd66fd0-a2d1-4de5-a353-acaa80749dd6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:28.809195 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695002d6-6b37-4a3a-b514-247111fcf604 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.814923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 466/930] 10.222.0.67 () {56 vars in 1126 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:28.815752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd66fd0-a2d1-4de5-a353-acaa80749dd6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.849239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f59f6-db38-45b3-a925-953d113b0748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.850144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f59f6-db38-45b3-a925-953d113b0748 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.850647 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f59f6-db38-45b3-a925-953d113b0748 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.851125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5f59f6-db38-45b3-a925-953d113b0748 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.851934 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f59f6-db38-45b3-a925-953d113b0748 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:28.852425 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f59f6-db38-45b3-a925-953d113b0748 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:28.853905 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5f59f6-db38-45b3-a925-953d113b0748 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:28.864164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 465/931] 10.222.0.67 () {56 vars in 1122 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:28.864164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f59f6-db38-45b3-a925-953d113b0748 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:28.931833 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd66fd0-a2d1-4de5-a353-acaa80749dd6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.967072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846b610-4e86-4df5-bd2c-0db9e20dc309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:28.967740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846b610-4e86-4df5-bd2c-0db9e20dc309 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:28.968026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846b610-4e86-4df5-bd2c-0db9e20dc309 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:28.968299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846b610-4e86-4df5-bd2c-0db9e20dc309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:28.979370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5f59f6-db38-45b3-a925-953d113b0748 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:28.988808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:28.990160 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846b610-4e86-4df5-bd2c-0db9e20dc309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:28.991622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846b610-4e86-4df5-bd2c-0db9e20dc309 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7846b610-4e86-4df5-bd2c-0db9e20dc309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:29.006674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:29.007872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:29.008935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:29.008935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:29.008935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:29.012681 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 467/932] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:29.013768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846b610-4e86-4df5-bd2c-0db9e20dc309 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:29.021783 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fecf1-e7d2-4bc9-aa6a-6be8cce0cc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:29.022397 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fecf1-e7d2-4bc9-aa6a-6be8cce0cc9a None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:29.022679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fecf1-e7d2-4bc9-aa6a-6be8cce0cc9a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:29.022951 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9fecf1-e7d2-4bc9-aa6a-6be8cce0cc9a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:29.024067 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fecf1-e7d2-4bc9-aa6a-6be8cce0cc9a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:29.025454 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9fecf1-e7d2-4bc9-aa6a-6be8cce0cc9a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:29.029812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:29.118763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846b610-4e86-4df5-bd2c-0db9e20dc309 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:29.153910 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 466/933] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 17:45:28 2022] POST /identity/v3/endpoints => generated 347 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:29.163103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59658d4b-585f-41b5-87ac-29d4d291abe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:29.163768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59658d4b-585f-41b5-87ac-29d4d291abe4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:29.164051 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59658d4b-585f-41b5-87ac-29d4d291abe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:29.164583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59658d4b-585f-41b5-87ac-29d4d291abe4 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:29.164841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fecf1-e7d2-4bc9-aa6a-6be8cce0cc9a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:29.165247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59658d4b-585f-41b5-87ac-29d4d291abe4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:29.165528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59658d4b-585f-41b5-87ac-29d4d291abe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:29.167063 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59658d4b-585f-41b5-87ac-29d4d291abe4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:29.182275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 468/934] 10.222.0.67 () {56 vars in 1122 bytes} [Mon Sep 26 17:45:29 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:29.189666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59658d4b-585f-41b5-87ac-29d4d291abe4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:29.305415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9fecf1-e7d2-4bc9-aa6a-6be8cce0cc9a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:29.308643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:29.356998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631358d-69de-4f32-bce1-cea60bc92d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:29.357228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59658d4b-585f-41b5-87ac-29d4d291abe4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:29.358096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631358d-69de-4f32-bce1-cea60bc92d80 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:29.358845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631358d-69de-4f32-bce1-cea60bc92d80 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:29.360227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f631358d-69de-4f32-bce1-cea60bc92d80 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:29.361154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631358d-69de-4f32-bce1-cea60bc92d80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:29.361462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631358d-69de-4f32-bce1-cea60bc92d80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:29.362836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f631358d-69de-4f32-bce1-cea60bc92d80 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:29.380676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 467/935] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:29.389759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631358d-69de-4f32-bce1-cea60bc92d80 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:29.408049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3b7ed-359d-46dc-bf01-8437a36046a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:29.408771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3b7ed-359d-46dc-bf01-8437a36046a2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:29.409056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3b7ed-359d-46dc-bf01-8437a36046a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:29.409333 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea3b7ed-359d-46dc-bf01-8437a36046a2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:29.410039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3b7ed-359d-46dc-bf01-8437a36046a2 None admin] RBAC: Adding query filter params (interface=public, service_id=c0b63082f5ad419db94abd88b5bdfc77, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:29.410346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3b7ed-359d-46dc-bf01-8437a36046a2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:29.411864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea3b7ed-359d-46dc-bf01-8437a36046a2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:29.417763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 469/936] 10.222.0.67 () {56 vars in 1251 bytes} [Mon Sep 26 17:45:29 2022] GET /identity/v3/endpoints?service_id=c0b63082f5ad419db94abd88b5bdfc77&interface=public®ion_id=RegionOne => generated 194 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:29.419894 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:29.423743 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3b7ed-359d-46dc-bf01-8437a36046a2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:29.518258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f631358d-69de-4f32-bce1-cea60bc92d80 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:29.556559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea3b7ed-359d-46dc-bf01-8437a36046a2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:29.559090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e194996-91cb-4e82-8922-85bd411deade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:29.560012 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e194996-91cb-4e82-8922-85bd411deade None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:29.560358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e194996-91cb-4e82-8922-85bd411deade None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:29.560699 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e194996-91cb-4e82-8922-85bd411deade None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:29.569638 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e194996-91cb-4e82-8922-85bd411deade None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:29.571299 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e194996-91cb-4e82-8922-85bd411deade None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8e194996-91cb-4e82-8922-85bd411deade None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:29.587038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:29.588059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:29.589346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:29.589346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:29.589346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:29.589346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:29.589346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:29.589346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:29.589346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:29.589346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:29.589346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:45:29.589346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:29.589346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 468/937] 10.222.0.67 () {56 vars in 1117 bytes} [Mon Sep 26 17:45:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:29.594924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e194996-91cb-4e82-8922-85bd411deade None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:29.605653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90debe08-8ca2-4a13-a3b5-9566cdeb42a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:29.606547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90debe08-8ca2-4a13-a3b5-9566cdeb42a6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:29.607074 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90debe08-8ca2-4a13-a3b5-9566cdeb42a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:29.607525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90debe08-8ca2-4a13-a3b5-9566cdeb42a6 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:29.608752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90debe08-8ca2-4a13-a3b5-9566cdeb42a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:29.611375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90debe08-8ca2-4a13-a3b5-9566cdeb42a6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:29.614594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:29.621724 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 470/938] 10.222.0.67 () {56 vars in 1087 bytes} [Mon Sep 26 17:45:29 2022] GET /identity/v3/services => generated 1497 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 17:45:29.701027 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:29.721399 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e194996-91cb-4e82-8922-85bd411deade None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:29.762416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75720612-cd0b-47eb-8d6c-f71bf80ea5ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:29.763135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75720612-cd0b-47eb-8d6c-f71bf80ea5ed None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:29.763753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75720612-cd0b-47eb-8d6c-f71bf80ea5ed None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:29.764106 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75720612-cd0b-47eb-8d6c-f71bf80ea5ed None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:29.764912 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75720612-cd0b-47eb-8d6c-f71bf80ea5ed None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:29.765288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75720612-cd0b-47eb-8d6c-f71bf80ea5ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:29.766957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75720612-cd0b-47eb-8d6c-f71bf80ea5ed None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:29.788046 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 469/939] 10.222.0.67 () {56 vars in 1149 bytes} [Mon Sep 26 17:45:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:29.797437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90debe08-8ca2-4a13-a3b5-9566cdeb42a6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:29.904483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90debe08-8ca2-4a13-a3b5-9566cdeb42a6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:29.940201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e7bd2-4684-4638-88c3-18b30c7ea35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:29.941097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e7bd2-4684-4638-88c3-18b30c7ea35e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:29.941535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e7bd2-4684-4638-88c3-18b30c7ea35e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:29.941949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721e7bd2-4684-4638-88c3-18b30c7ea35e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:29.953715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e7bd2-4684-4638-88c3-18b30c7ea35e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:29.955489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721e7bd2-4684-4638-88c3-18b30c7ea35e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-721e7bd2-4684-4638-88c3-18b30c7ea35e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:29.971273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:29.972443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:29.973512 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:29.973512 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:29.973512 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:29.975183 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 471/940] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:29.985391 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75720612-cd0b-47eb-8d6c-f71bf80ea5ed None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:30.096107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75720612-cd0b-47eb-8d6c-f71bf80ea5ed None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:30.133700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5232b8-9789-41aa-8254-6d53fd7084a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:30.134778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5232b8-9789-41aa-8254-6d53fd7084a5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:30.135284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5232b8-9789-41aa-8254-6d53fd7084a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:30.135797 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5232b8-9789-41aa-8254-6d53fd7084a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:30.136658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5232b8-9789-41aa-8254-6d53fd7084a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:30.137178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5232b8-9789-41aa-8254-6d53fd7084a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:30.139002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5232b8-9789-41aa-8254-6d53fd7084a5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:30.148987 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:30.154451 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 470/941] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:30.160919 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e7bd2-4684-4638-88c3-18b30c7ea35e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:30.229657 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:30.282086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721e7bd2-4684-4638-88c3-18b30c7ea35e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:30.317730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab523971-2bbf-4857-8728-a277812da6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:30.318350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab523971-2bbf-4857-8728-a277812da6ad None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:30.318633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab523971-2bbf-4857-8728-a277812da6ad None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:30.318910 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab523971-2bbf-4857-8728-a277812da6ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:30.325891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab523971-2bbf-4857-8728-a277812da6ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:30.327339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab523971-2bbf-4857-8728-a277812da6ad None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:30.339680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ab523971-2bbf-4857-8728-a277812da6ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:30.339680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:30.339680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:30.339680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:30.339680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:30.339680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:30.339680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:30.339680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:30.339680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:30.339680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:30.339680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:30.339680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:30.339680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:30.339680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:30.339680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:30.340567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:30.341437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:30.341437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:30.341437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:30.341437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:30.341437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:30.341437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:30.341437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:30.341437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:30.341437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 472/942] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:30.348926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5232b8-9789-41aa-8254-6d53fd7084a5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:30.455001 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5232b8-9789-41aa-8254-6d53fd7084a5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:30.492109 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a1d72c-193e-4995-ad0e-d60fe787dbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:30.493102 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1d72c-193e-4995-ad0e-d60fe787dbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:30.493590 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1d72c-193e-4995-ad0e-d60fe787dbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:30.494065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a1d72c-193e-4995-ad0e-d60fe787dbf3 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:30.494992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1d72c-193e-4995-ad0e-d60fe787dbf3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:30.495500 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1d72c-193e-4995-ad0e-d60fe787dbf3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:30.497147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a1d72c-193e-4995-ad0e-d60fe787dbf3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:30.510243 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 471/943] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:30.516774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab523971-2bbf-4857-8728-a277812da6ad None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:30.633099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab523971-2bbf-4857-8728-a277812da6ad None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:30.647149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:30.668316 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72734501-3a3c-42c9-baf6-2b76c09c3483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:30.668979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72734501-3a3c-42c9-baf6-2b76c09c3483 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:30.669315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72734501-3a3c-42c9-baf6-2b76c09c3483 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:30.669598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72734501-3a3c-42c9-baf6-2b76c09c3483 None admin] PATH_INFO: `/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/1d201727ffae42c4bd704d26c03c5d9b/roles/fb6eaf230a474167b3a3726cbd2d35aa` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:30.672499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72734501-3a3c-42c9-baf6-2b76c09c3483 None admin] RBAC: Authorizing `identity:create_grant(project_id=71c96e5caeb44927bc21504fe54f0f98, user_id=1d201727ffae42c4bd704d26c03c5d9b, role_id=fb6eaf230a474167b3a3726cbd2d35aa)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:30.674083 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72734501-3a3c-42c9-baf6-2b76c09c3483 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:30.706571 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213e83b-eb4a-40c9-95a2-8970dcef4987 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:30.706859 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213e83b-eb4a-40c9-95a2-8970dcef4987 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:30.707132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213e83b-eb4a-40c9-95a2-8970dcef4987 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:30.708665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 472/944] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:30.713809 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4277c-a0c1-4b05-a1b9-b585e0be35a8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:30.714090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4277c-a0c1-4b05-a1b9-b585e0be35a8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:30.714362 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4277c-a0c1-4b05-a1b9-b585e0be35a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:30.733169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 473/945] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:45:30 2022] PUT /identity/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/1d201727ffae42c4bd704d26c03c5d9b/roles/fb6eaf230a474167b3a3726cbd2d35aa => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:30.738078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9ac4277c-a0c1-4b05-a1b9-b585e0be35a8 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:30.739310 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9ac4277c-a0c1-4b05-a1b9-b585e0be35a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:30.808267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:30.821005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 473/946] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:31.103304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72734501-3a3c-42c9-baf6-2b76c09c3483 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:31.212273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72734501-3a3c-42c9-baf6-2b76c09c3483 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:31.249182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a74deda-3f54-46c3-8cb7-124714fca1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:31.249741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74deda-3f54-46c3-8cb7-124714fca1da None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:31.250073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74deda-3f54-46c3-8cb7-124714fca1da None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:31.250456 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a74deda-3f54-46c3-8cb7-124714fca1da None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:31.251190 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74deda-3f54-46c3-8cb7-124714fca1da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:31.252664 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a74deda-3f54-46c3-8cb7-124714fca1da None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:31.255287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: WARNING keystone.server.flask.common [None req-5a74deda-3f54-46c3-8cb7-124714fca1da None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 26 17:45:31.318455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.sql.core [None req-5a74deda-3f54-46c3-8cb7-124714fca1da None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 26 17:45:31.318455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 26 17:45:31.318455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [parameters: {'id': '0ab879802bb24eec82fca48c97c92a33', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 26 17:45:31.318455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81500) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 26 17:45:31.336069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5a74deda-3f54-46c3-8cb7-124714fca1da None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0ab879802bb24eec82fca48c97c92a33.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0ab879802bb24eec82fca48c97c92a33. Sep 26 17:45:31.336069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:31.336069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 17:45:31.336069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 17:45:31.336069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 17:45:31.336069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 17:45:31.336069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 17:45:31.336069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 17:45:31.336069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 17:45:31.336069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Sep 26 17:45:31.336069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 17:45:31.336069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 17:45:31.336069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 26 17:45:31.336934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application yield resource Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.rollback() Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.force_reraise() Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise self.value Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application session.commit() Sep 26 17:45:31.337890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._prepare_impl() Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.flush() Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._flush(objects) Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application compat.raise_( Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application flush_context.execute() Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 26 17:45:31.338770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rec.execute(self) Sep 26 17:45:31.339672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 26 17:45:31.339672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 26 17:45:31.339672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 26 17:45:31.339672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 26 17:45:31.339672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 26 17:45:31.339672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 26 17:45:31.339672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 26 17:45:31.339672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 26 17:45:31.339672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 26 17:45:31.339672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 26 17:45:31.339672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 26 17:45:31.339672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 26 17:45:31.339672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 26 17:45:31.339672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 26 17:45:31.339672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 17:45:31.340471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 17:45:31.341315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 17:45:31.341315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 17:45:31.341315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 17:45:31.341315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 17:45:31.341315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 17:45:31.341315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 17:45:31.341315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 26 17:45:31.341315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 26 17:45:31.341315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [parameters: {'id': '0ab879802bb24eec82fca48c97c92a33', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 26 17:45:31.341315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 26 17:45:31.341315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:31.341315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:31.341315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:31.341315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:31.341315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:31.342066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:31.342950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:31.342950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 26 17:45:31.342950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 26 17:45:31.342950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:31.342950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:31.342950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 26 17:45:31.342950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 26 17:45:31.342950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0ab879802bb24eec82fca48c97c92a33. Sep 26 17:45:31.342950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:31.342950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 474/947] 10.222.0.67 () {60 vars in 1140 bytes} [Mon Sep 26 17:45:31 2022] POST /identity/v3/projects => generated 262 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:31.353172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4277c-a0c1-4b05-a1b9-b585e0be35a8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:31.463991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4277c-a0c1-4b05-a1b9-b585e0be35a8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:31.501300 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb87d1c-b903-4e72-9ae0-2eef2132cad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:31.501876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb87d1c-b903-4e72-9ae0-2eef2132cad4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:31.502159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb87d1c-b903-4e72-9ae0-2eef2132cad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:31.502434 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb87d1c-b903-4e72-9ae0-2eef2132cad4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:31.509385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd756ad-8f92-4cfa-afee-f640a6ed9636 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:31.509669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd756ad-8f92-4cfa-afee-f640a6ed9636 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:31.509940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd756ad-8f92-4cfa-afee-f640a6ed9636 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:31.514297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb87d1c-b903-4e72-9ae0-2eef2132cad4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:31.515041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 475/948] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:31.516766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb87d1c-b903-4e72-9ae0-2eef2132cad4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:31.521530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2ed79-9285-4418-9d9e-8e9483dd10be None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:31.521890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2ed79-9285-4418-9d9e-8e9483dd10be None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:31.522240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2ed79-9285-4418-9d9e-8e9483dd10be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:31.531402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-deb87d1c-b903-4e72-9ae0-2eef2132cad4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:31.531402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:31.531402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:31.531402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:31.531402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:31.531402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:31.531402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:31.531402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:31.531402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:31.531402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:31.531402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:31.531402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:31.531402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:31.531402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:31.531402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:31.532361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:31.533215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:31.533215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:31.533215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:31.533215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:31.533215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:31.533215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:31.533215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:31.533215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:31.535336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 474/949] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:31.541726 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb87d1c-b903-4e72-9ae0-2eef2132cad4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:31.546251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3ad2ed79-9285-4418-9d9e-8e9483dd10be None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:31.546251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3ad2ed79-9285-4418-9d9e-8e9483dd10be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:31.571082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:31.607776 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:31.627957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 476/950] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:31.666508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb87d1c-b903-4e72-9ae0-2eef2132cad4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:31.702922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b4d51-83cb-47fb-9aa9-a3c7e82cd552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:31.703808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b4d51-83cb-47fb-9aa9-a3c7e82cd552 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:31.704237 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b4d51-83cb-47fb-9aa9-a3c7e82cd552 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:31.704659 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b4d51-83cb-47fb-9aa9-a3c7e82cd552 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:31.705482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b4d51-83cb-47fb-9aa9-a3c7e82cd552 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:31.705930 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b4d51-83cb-47fb-9aa9-a3c7e82cd552 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:31.707499 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018b4d51-83cb-47fb-9aa9-a3c7e82cd552 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:31.727547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 475/951] 10.222.0.67 () {56 vars in 1112 bytes} [Mon Sep 26 17:45:31 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:31.899850 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2ed79-9285-4418-9d9e-8e9483dd10be None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:31.952296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07428fbe-98f4-4899-b7ad-f1d8c6c21c43 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:31.952669 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07428fbe-98f4-4899-b7ad-f1d8c6c21c43 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:31.953417 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07428fbe-98f4-4899-b7ad-f1d8c6c21c43 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:31.955414 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 476/952] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:31.961897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c40f8-9b23-4219-b5e5-6c8dbf6c4139 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:31.962180 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c40f8-9b23-4219-b5e5-6c8dbf6c4139 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:31.962456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c40f8-9b23-4219-b5e5-6c8dbf6c4139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:31.986865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f73c40f8-9b23-4219-b5e5-6c8dbf6c4139 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:31.988811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f73c40f8-9b23-4219-b5e5-6c8dbf6c4139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:32.019875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2ed79-9285-4418-9d9e-8e9483dd10be None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:32.060812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb426add-40ee-430a-a93f-25bdbc46cac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:32.061553 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb426add-40ee-430a-a93f-25bdbc46cac1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:32.061840 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb426add-40ee-430a-a93f-25bdbc46cac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:32.062111 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb426add-40ee-430a-a93f-25bdbc46cac1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:32.062949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb426add-40ee-430a-a93f-25bdbc46cac1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:32.064877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb426add-40ee-430a-a93f-25bdbc46cac1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:32.066963 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 477/953] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-eb426add-40ee-430a-a93f-25bdbc46cac1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:32.080400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:32.081394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:32.082203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:32.082203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:32.082203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:32.082203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:32.082203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:32.082203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:32.082203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:32.082203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:32.082203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:32.082203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:32.082203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:32.082203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:32.082203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:45:32.082203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:32.082203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 477/954] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:31 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:32.087452 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c40f8-9b23-4219-b5e5-6c8dbf6c4139 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:32.129833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:32.193449 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c40f8-9b23-4219-b5e5-6c8dbf6c4139 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:32.233000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55b838-7d7a-43ff-b637-2fd6611964b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:32.233612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55b838-7d7a-43ff-b637-2fd6611964b3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:32.233896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55b838-7d7a-43ff-b637-2fd6611964b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:32.234826 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e55b838-7d7a-43ff-b637-2fd6611964b3 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:32.235759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55b838-7d7a-43ff-b637-2fd6611964b3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:32.236067 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55b838-7d7a-43ff-b637-2fd6611964b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:32.237486 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e55b838-7d7a-43ff-b637-2fd6611964b3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:32.242669 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 478/955] 10.222.0.67 () {56 vars in 1112 bytes} [Mon Sep 26 17:45:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:32.248100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb426add-40ee-430a-a93f-25bdbc46cac1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:32.274768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:32.356778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb426add-40ee-430a-a93f-25bdbc46cac1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:32.373410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55b838-7d7a-43ff-b637-2fd6611964b3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:32.396295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bf61c-f619-47f2-bea5-60e7376908bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:32.396915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bf61c-f619-47f2-bea5-60e7376908bc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:32.397642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bf61c-f619-47f2-bea5-60e7376908bc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:32.397928 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7bf61c-f619-47f2-bea5-60e7376908bc None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:32.398850 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bf61c-f619-47f2-bea5-60e7376908bc None admin] RBAC: Adding query filter params (type=compute) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:32.399162 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bf61c-f619-47f2-bea5-60e7376908bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:32.401039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7bf61c-f619-47f2-bea5-60e7376908bc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:32.408829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 478/956] 10.222.0.67 () {56 vars in 1112 bytes} [Mon Sep 26 17:45:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:32.489838 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e55b838-7d7a-43ff-b637-2fd6611964b3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:32.492618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:32.528757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fda25d-175c-4d35-858a-e2cfc531c347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:32.529416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fda25d-175c-4d35-858a-e2cfc531c347 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:32.529701 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fda25d-175c-4d35-858a-e2cfc531c347 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:32.529977 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fda25d-175c-4d35-858a-e2cfc531c347 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:32.530643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fda25d-175c-4d35-858a-e2cfc531c347 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:32.532666 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fda25d-175c-4d35-858a-e2cfc531c347 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-d8fda25d-175c-4d35-858a-e2cfc531c347 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:32.542645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:32.543867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:32.544717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:32.544717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:32.544717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:32.544717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:32.544717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:32.544717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:32.544717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:32.544717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:32.544717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:32.544717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:32.544717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:32.544717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:32.544717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:45:32.544717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:32.544717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 479/957] 10.222.0.67 () {56 vars in 1126 bytes} [Mon Sep 26 17:45:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:32.552827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bf61c-f619-47f2-bea5-60e7376908bc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:32.651322 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:32.667940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7bf61c-f619-47f2-bea5-60e7376908bc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:32.710516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dbe7e-4071-4e60-93e8-cf53b1dadf30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:32.711126 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dbe7e-4071-4e60-93e8-cf53b1dadf30 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:32.711454 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dbe7e-4071-4e60-93e8-cf53b1dadf30 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:32.711756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dbe7e-4071-4e60-93e8-cf53b1dadf30 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:32.712432 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20dbe7e-4071-4e60-93e8-cf53b1dadf30 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:32.712736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20dbe7e-4071-4e60-93e8-cf53b1dadf30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:32.714103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20dbe7e-4071-4e60-93e8-cf53b1dadf30 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:32.719710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 479/958] 10.222.0.67 () {56 vars in 1122 bytes} [Mon Sep 26 17:45:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:32.728742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fda25d-175c-4d35-858a-e2cfc531c347 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:32.845035 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fda25d-175c-4d35-858a-e2cfc531c347 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:32.881210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900889e7-5023-4bb2-b652-3e8073ca3e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:32.881822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900889e7-5023-4bb2-b652-3e8073ca3e1f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:32.882099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900889e7-5023-4bb2-b652-3e8073ca3e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:32.882367 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900889e7-5023-4bb2-b652-3e8073ca3e1f None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:32.882993 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900889e7-5023-4bb2-b652-3e8073ca3e1f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:32.883292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900889e7-5023-4bb2-b652-3e8073ca3e1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:32.884683 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900889e7-5023-4bb2-b652-3e8073ca3e1f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:32.890871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 480/959] 10.222.0.67 () {56 vars in 1122 bytes} [Mon Sep 26 17:45:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:32.896666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dbe7e-4071-4e60-93e8-cf53b1dadf30 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:33.008300 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dbe7e-4071-4e60-93e8-cf53b1dadf30 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:33.052788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a83236f-f868-4485-b588-ca34c06e3a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:33.053402 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83236f-f868-4485-b588-ca34c06e3a98 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:33.053682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83236f-f868-4485-b588-ca34c06e3a98 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:33.053954 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a83236f-f868-4485-b588-ca34c06e3a98 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:33.054632 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83236f-f868-4485-b588-ca34c06e3a98 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:33.056040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a83236f-f868-4485-b588-ca34c06e3a98 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:33.126172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 480/960] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 17:45:32 2022] POST /identity/v3/endpoints => generated 349 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:33.136888 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001647ec-d0a2-4322-b8db-f01495ead054 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:33.137223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001647ec-d0a2-4322-b8db-f01495ead054 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:33.137607 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001647ec-d0a2-4322-b8db-f01495ead054 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:33.139190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 481/961] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:33.146086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758e413-36f2-426c-894d-ebe81ee0e195 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:33.146404 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758e413-36f2-426c-894d-ebe81ee0e195 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:33.146645 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758e413-36f2-426c-894d-ebe81ee0e195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:33.162752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:33.170024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7758e413-36f2-426c-894d-ebe81ee0e195 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:33.171311 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7758e413-36f2-426c-894d-ebe81ee0e195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:33.263072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 481/962] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:33.532294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001647ec-d0a2-4322-b8db-f01495ead054 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:33.567566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:33.644178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001647ec-d0a2-4322-b8db-f01495ead054 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:33.679830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b703b3-980e-4826-af35-e6ffb8a7f2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:33.680470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b703b3-980e-4826-af35-e6ffb8a7f2dd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:33.680751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b703b3-980e-4826-af35-e6ffb8a7f2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:33.681025 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b703b3-980e-4826-af35-e6ffb8a7f2dd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:33.692817 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b703b3-980e-4826-af35-e6ffb8a7f2dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e9b703b3-980e-4826-af35-e6ffb8a7f2dd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:33.697704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:33.699770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:33.701682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:33.701682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:33.701682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:33.701682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:33.701682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:33.701682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:33.701682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:33.701682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:33.701682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:33.701682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:33.701682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:45:33.701682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:33.707743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 482/963] 10.222.0.67 () {56 vars in 1120 bytes} [Mon Sep 26 17:45:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:33.707935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758e413-36f2-426c-894d-ebe81ee0e195 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:33.809533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758e413-36f2-426c-894d-ebe81ee0e195 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:33.845385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:33.846963 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1849f53-2199-4101-96ca-e8e7b31b9b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:33.847583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1849f53-2199-4101-96ca-e8e7b31b9b3d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:33.847888 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1849f53-2199-4101-96ca-e8e7b31b9b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:33.848159 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1849f53-2199-4101-96ca-e8e7b31b9b3d None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:33.848889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1849f53-2199-4101-96ca-e8e7b31b9b3d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:33.849192 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1849f53-2199-4101-96ca-e8e7b31b9b3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:33.850563 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1849f53-2199-4101-96ca-e8e7b31b9b3d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:33.859436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 482/964] 10.222.0.67 () {56 vars in 1115 bytes} [Mon Sep 26 17:45:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:33.865643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b703b3-980e-4826-af35-e6ffb8a7f2dd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:33.964127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:33.972841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b703b3-980e-4826-af35-e6ffb8a7f2dd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:34.011291 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05e5141-10ea-44b6-b315-098498344c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:34.011980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e5141-10ea-44b6-b315-098498344c56 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.012260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e5141-10ea-44b6-b315-098498344c56 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.012531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05e5141-10ea-44b6-b315-098498344c56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.027749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e5141-10ea-44b6-b315-098498344c56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:34.029220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05e5141-10ea-44b6-b315-098498344c56 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a05e5141-10ea-44b6-b315-098498344c56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:34.050514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:34.051552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:34.052456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:34.052456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:34.052456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:34.055881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 483/965] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:34.059928 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1849f53-2199-4101-96ca-e8e7b31b9b3d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:34.124878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec791c-531e-4839-b89e-c7c0225042bc None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.125216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec791c-531e-4839-b89e-c7c0225042bc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.125409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec791c-531e-4839-b89e-c7c0225042bc None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.127117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 484/966] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:34.132144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb05ae-67ce-4df8-bc12-4a51635894a8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.132432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb05ae-67ce-4df8-bc12-4a51635894a8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.132699 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afb05ae-67ce-4df8-bc12-4a51635894a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.155464 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7afb05ae-67ce-4df8-bc12-4a51635894a8 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:34.157598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7afb05ae-67ce-4df8-bc12-4a51635894a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:34.173326 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1849f53-2199-4101-96ca-e8e7b31b9b3d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:34.209624 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff106f5-0122-4be5-be94-a4c6adff151b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:34.210628 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff106f5-0122-4be5-be94-a4c6adff151b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.211155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff106f5-0122-4be5-be94-a4c6adff151b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.211633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff106f5-0122-4be5-be94-a4c6adff151b None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.212548 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff106f5-0122-4be5-be94-a4c6adff151b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:34.213054 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff106f5-0122-4be5-be94-a4c6adff151b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:34.214632 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff106f5-0122-4be5-be94-a4c6adff151b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:34.228638 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 483/967] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:34.234939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff106f5-0122-4be5-be94-a4c6adff151b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:34.251915 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 485/968] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:34.294293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:34.342650 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff106f5-0122-4be5-be94-a4c6adff151b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:34.381340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a786e-e572-4009-8082-81648fc1195a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:34.381981 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a786e-e572-4009-8082-81648fc1195a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.382260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a786e-e572-4009-8082-81648fc1195a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.382532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a786e-e572-4009-8082-81648fc1195a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.390437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a786e-e572-4009-8082-81648fc1195a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:34.391908 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a786e-e572-4009-8082-81648fc1195a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2c9a786e-e572-4009-8082-81648fc1195a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:34.403937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:34.404914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:34.406033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:34.406033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:34.406033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:34.406033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:34.406033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:34.406033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:34.406033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:34.406033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:34.406033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 26 17:45:34.406033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:34.410165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 484/969] 10.222.0.67 () {56 vars in 1117 bytes} [Mon Sep 26 17:45:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:34.411532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb05ae-67ce-4df8-bc12-4a51635894a8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:34.491399 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:34.518503 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afb05ae-67ce-4df8-bc12-4a51635894a8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:34.551922 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a786e-e572-4009-8082-81648fc1195a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:34.557926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdf903-cc86-4b08-bff5-8246cceff9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:34.559178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdf903-cc86-4b08-bff5-8246cceff9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.559690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdf903-cc86-4b08-bff5-8246cceff9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.560203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfdf903-cc86-4b08-bff5-8246cceff9c2 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.561196 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfdf903-cc86-4b08-bff5-8246cceff9c2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:34.561762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfdf903-cc86-4b08-bff5-8246cceff9c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:34.563996 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfdf903-cc86-4b08-bff5-8246cceff9c2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:34.583218 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 486/970] 10.222.0.67 () {56 vars in 1149 bytes} [Mon Sep 26 17:45:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:34.590957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdf903-cc86-4b08-bff5-8246cceff9c2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:34.678858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a786e-e572-4009-8082-81648fc1195a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:34.695878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:34.710721 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfdf903-cc86-4b08-bff5-8246cceff9c2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:34.716717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8d936-ce22-4100-acd7-69580f0550c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:34.717321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8d936-ce22-4100-acd7-69580f0550c4 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.717603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8d936-ce22-4100-acd7-69580f0550c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.717878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f8d936-ce22-4100-acd7-69580f0550c4 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.719136 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8d936-ce22-4100-acd7-69580f0550c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:34.720928 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f8d936-ce22-4100-acd7-69580f0550c4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:34.746405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:34.751772 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3adb2ff-15d6-46bb-a5c0-2619d94208c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:34.752391 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adb2ff-15d6-46bb-a5c0-2619d94208c0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.752674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adb2ff-15d6-46bb-a5c0-2619d94208c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.752947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3adb2ff-15d6-46bb-a5c0-2619d94208c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.765837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.sql.core [None req-89f8d936-ce22-4100-acd7-69580f0550c4 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 26 17:45:34.765837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 26 17:45:34.765837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [parameters: {'user_id': 'c8b31c74199f4bb197f0f88ab2545069', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 26 17:45:34.765837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81500) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 26 17:45:34.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-89f8d936-ce22-4100-acd7-69580f0550c4 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 26 17:45:34.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:34.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 17:45:34.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 17:45:34.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 17:45:34.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 17:45:34.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 17:45:34.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 17:45:34.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 17:45:34.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Sep 26 17:45:34.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 17:45:34.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 17:45:34.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 17:45:34.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Sep 26 17:45:34.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 17:45:34.777307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application yield resource Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.rollback() Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.force_reraise() Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise self.value Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application session.commit() Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 26 17:45:34.778306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._prepare_impl() Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.flush() Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._flush(objects) Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application compat.raise_( Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application flush_context.execute() Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rec.execute(self) Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 26 17:45:34.779209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 26 17:45:34.780114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 26 17:45:34.782715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 26 17:45:34.784717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 26 17:45:34.784717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 26 17:45:34.784717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 26 17:45:34.784717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 26 17:45:34.784717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 26 17:45:34.784717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 26 17:45:34.784717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c8b31c74199f4bb197f0f88ab2545069', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 26 17:45:34.784717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 26 17:45:34.784717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:34.784717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:34.784717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:34.784717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:34.784717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:34.784717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:34.784717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 26 17:45:34.787122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 26 17:45:34.789355 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 26 17:45:34.789355 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 26 17:45:34.789355 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 26 17:45:34.789355 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 26 17:45:34.789355 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 26 17:45:34.789355 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:34.789355 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 485/971] 10.222.0.67 () {60 vars in 1131 bytes} [Mon Sep 26 17:45:34 2022] POST /identity/v3/users => generated 160 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:34.790334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8d936-ce22-4100-acd7-69580f0550c4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:34.837455 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3adb2ff-15d6-46bb-a5c0-2619d94208c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:34.838894 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3adb2ff-15d6-46bb-a5c0-2619d94208c0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:34.900804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f8d936-ce22-4100-acd7-69580f0550c4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:34.936475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e96cabd-7fb6-4db5-a617-24979193a6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:34.937214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96cabd-7fb6-4db5-a617-24979193a6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.937563 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96cabd-7fb6-4db5-a617-24979193a6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.937976 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e96cabd-7fb6-4db5-a617-24979193a6e6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.949030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 487/972] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:34 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 17:45:34.949030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d13ba-c207-4447-b8e1-668f4025fd92 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.949030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d13ba-c207-4447-b8e1-668f4025fd92 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.949030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d13ba-c207-4447-b8e1-668f4025fd92 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.949030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 488/973] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:34.950401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d13ba-c207-4447-b8e1-668f4025fd92 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:34.953040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96cabd-7fb6-4db5-a617-24979193a6e6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:34.954715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e96cabd-7fb6-4db5-a617-24979193a6e6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8e96cabd-7fb6-4db5-a617-24979193a6e6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:34.966801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:34.967968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:34.969688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:34.969688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:34.969688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:34.969688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:34.969688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:34.969688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:34.969688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:34.969688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:34.969688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 26 17:45:34.969688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:34.973531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 486/974] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:34.974772 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8352941-c65c-4553-8723-2b0679bad7bd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:34.975052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8352941-c65c-4553-8723-2b0679bad7bd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:34.975330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8352941-c65c-4553-8723-2b0679bad7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:34.993411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:34.996359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a8352941-c65c-4553-8723-2b0679bad7bd None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:34.997939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a8352941-c65c-4553-8723-2b0679bad7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:35.028650 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:35.070860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 487/975] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:35.072144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8352941-c65c-4553-8723-2b0679bad7bd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:35.082392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d13ba-c207-4447-b8e1-668f4025fd92 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:35.129614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0e31f-4bc0-413e-b285-ca91c00173f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:35.130557 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0e31f-4bc0-413e-b285-ca91c00173f8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.131021 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0e31f-4bc0-413e-b285-ca91c00173f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.132024 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d0e31f-4bc0-413e-b285-ca91c00173f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.145284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d0e31f-4bc0-413e-b285-ca91c00173f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:35.147791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d0e31f-4bc0-413e-b285-ca91c00173f8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-11d0e31f-4bc0-413e-b285-ca91c00173f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:35.165640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:35.166668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:35.167534 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:35.167534 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:35.167534 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:35.169612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 489/976] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:35.177612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0e31f-4bc0-413e-b285-ca91c00173f8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:35.186607 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8352941-c65c-4553-8723-2b0679bad7bd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:35.226812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34499ff-21c7-42e8-8713-11857e7f1c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:35.227587 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34499ff-21c7-42e8-8713-11857e7f1c3f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.227967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34499ff-21c7-42e8-8713-11857e7f1c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.228312 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34499ff-21c7-42e8-8713-11857e7f1c3f None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.229537 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34499ff-21c7-42e8-8713-11857e7f1c3f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:35.229922 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34499ff-21c7-42e8-8713-11857e7f1c3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:35.231987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34499ff-21c7-42e8-8713-11857e7f1c3f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:35.282673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:35.295684 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 488/977] 10.222.0.67 () {56 vars in 1101 bytes} [Mon Sep 26 17:45:35 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:35.302950 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d0e31f-4bc0-413e-b285-ca91c00173f8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:35.342913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c695a-eead-4958-8cc5-116b4be37b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:35.343665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c695a-eead-4958-8cc5-116b4be37b45 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.343934 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c695a-eead-4958-8cc5-116b4be37b45 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.344211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c695a-eead-4958-8cc5-116b4be37b45 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.344967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c695a-eead-4958-8cc5-116b4be37b45 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:35.345271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c695a-eead-4958-8cc5-116b4be37b45 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:35.346765 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c695a-eead-4958-8cc5-116b4be37b45 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:35.362419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 490/978] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:35.363379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45353737-b907-4b5b-ae95-15648937a5e2 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.363711 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45353737-b907-4b5b-ae95-15648937a5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.363989 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45353737-b907-4b5b-ae95-15648937a5e2 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.365429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 489/979] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:35.368056 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c695a-eead-4958-8cc5-116b4be37b45 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:35.370610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45353737-b907-4b5b-ae95-15648937a5e2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:35.399322 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:35.485868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c695a-eead-4958-8cc5-116b4be37b45 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:35.493305 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45353737-b907-4b5b-ae95-15648937a5e2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:35.524991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f3fe9-a59d-4cbd-acac-7d1d03650003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:35.525624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f3fe9-a59d-4cbd-acac-7d1d03650003 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.525903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f3fe9-a59d-4cbd-acac-7d1d03650003 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.526173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f3fe9-a59d-4cbd-acac-7d1d03650003 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.532995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471729d7-760c-4851-84f5-62d59706414b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:35.533451 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f3fe9-a59d-4cbd-acac-7d1d03650003 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:35.533648 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471729d7-760c-4851-84f5-62d59706414b None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.533881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471729d7-760c-4851-84f5-62d59706414b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.534151 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471729d7-760c-4851-84f5-62d59706414b None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.534797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471729d7-760c-4851-84f5-62d59706414b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:35.534896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f3fe9-a59d-4cbd-acac-7d1d03650003 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:35.536418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471729d7-760c-4851-84f5-62d59706414b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:35.547223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-407f3fe9-a59d-4cbd-acac-7d1d03650003 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:35.547223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:35.547223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:35.547223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:35.547223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:35.547223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:35.547223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:35.547223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:35.547223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:35.547223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:35.547223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:35.547223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:35.547223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:35.547223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:35.547223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:35.550395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:35.551336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:35.551336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:35.551336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:35.551336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:35.551336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:35.551336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:35.551336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:35.551336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:35.551336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 491/980] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:35.551336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebc0c8-58ce-4e91-9a8c-5c0d3acb3c50 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.551336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebc0c8-58ce-4e91-9a8c-5c0d3acb3c50 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.551953 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cebc0c8-58ce-4e91-9a8c-5c0d3acb3c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.558785 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:35.570937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7cebc0c8-58ce-4e91-9a8c-5c0d3acb3c50 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:35.572204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7cebc0c8-58ce-4e91-9a8c-5c0d3acb3c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:35.638872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 492/981] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:35.639743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cebc0c8-58ce-4e91-9a8c-5c0d3acb3c50 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:35.645661 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 490/982] 10.222.0.67 () {60 vars in 1141 bytes} [Mon Sep 26 17:45:35 2022] POST /identity/v3/services => generated 242 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:45:35.756217 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cebc0c8-58ce-4e91-9a8c-5c0d3acb3c50 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:35.795576 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:35.802946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbced05-11ef-415e-8b8a-2112da1d258f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:35.803569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbced05-11ef-415e-8b8a-2112da1d258f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.803873 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbced05-11ef-415e-8b8a-2112da1d258f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.804144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbced05-11ef-415e-8b8a-2112da1d258f None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.804826 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbced05-11ef-415e-8b8a-2112da1d258f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:35.805126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbced05-11ef-415e-8b8a-2112da1d258f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:35.806554 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbced05-11ef-415e-8b8a-2112da1d258f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:35.819637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 493/983] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:35.825499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471729d7-760c-4851-84f5-62d59706414b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:35.911252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbced05-11ef-415e-8b8a-2112da1d258f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:35.914113 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:35.930778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471729d7-760c-4851-84f5-62d59706414b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:35.970755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b565f7-07c0-4c3e-86f4-82f50c11a937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:35.971377 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b565f7-07c0-4c3e-86f4-82f50c11a937 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:35.971675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b565f7-07c0-4c3e-86f4-82f50c11a937 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:35.971964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b565f7-07c0-4c3e-86f4-82f50c11a937 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:35.972698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b565f7-07c0-4c3e-86f4-82f50c11a937 None admin] RBAC: Adding query filter params (role.id=fb6eaf230a474167b3a3726cbd2d35aa, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=1d201727ffae42c4bd704d26c03c5d9b) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:35.973073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b565f7-07c0-4c3e-86f4-82f50c11a937 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:35.974562 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b565f7-07c0-4c3e-86f4-82f50c11a937 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:35.987870 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 491/984] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:35 2022] GET /identity/v3/role_assignments?user.id=1d201727ffae42c4bd704d26c03c5d9b&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=fb6eaf230a474167b3a3726cbd2d35aa => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:36.035154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbced05-11ef-415e-8b8a-2112da1d258f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:36.073226 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aca00a-cd8d-49cb-8b9b-bdff26bc2168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:36.073871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aca00a-cd8d-49cb-8b9b-bdff26bc2168 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:36.074147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aca00a-cd8d-49cb-8b9b-bdff26bc2168 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:36.074418 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aca00a-cd8d-49cb-8b9b-bdff26bc2168 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:36.075000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aca00a-cd8d-49cb-8b9b-bdff26bc2168 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:36.076421 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aca00a-cd8d-49cb-8b9b-bdff26bc2168 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-36aca00a-cd8d-49cb-8b9b-bdff26bc2168 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:36.086470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:36.087487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:36.092711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:36.092711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:36.092711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:36.092711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:36.092711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:36.092711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:36.092711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:36.092711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:36.092711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:36.092711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:36.092711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:36.092711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:36.092711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:45:36.092711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:36.092711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 494/985] 10.222.0.67 () {56 vars in 1126 bytes} [Mon Sep 26 17:45:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:36.095087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b565f7-07c0-4c3e-86f4-82f50c11a937 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:36.207587 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b565f7-07c0-4c3e-86f4-82f50c11a937 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:36.243088 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7641b85-44f7-40ca-a035-2762f888f316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:36.243730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7641b85-44f7-40ca-a035-2762f888f316 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:36.244012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7641b85-44f7-40ca-a035-2762f888f316 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:36.244285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7641b85-44f7-40ca-a035-2762f888f316 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:36.244913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7641b85-44f7-40ca-a035-2762f888f316 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:36.245215 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7641b85-44f7-40ca-a035-2762f888f316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:36.246572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7641b85-44f7-40ca-a035-2762f888f316 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:36.252592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 492/986] 10.222.0.67 () {56 vars in 1122 bytes} [Mon Sep 26 17:45:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:36.258102 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aca00a-cd8d-49cb-8b9b-bdff26bc2168 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:36.317969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:36.365992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aca00a-cd8d-49cb-8b9b-bdff26bc2168 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:36.401994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c64bfa-1a99-443f-9592-277e30f71ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:36.402608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c64bfa-1a99-443f-9592-277e30f71ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:36.402887 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c64bfa-1a99-443f-9592-277e30f71ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:36.403159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c64bfa-1a99-443f-9592-277e30f71ec8 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:36.403816 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c64bfa-1a99-443f-9592-277e30f71ec8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:36.404121 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c64bfa-1a99-443f-9592-277e30f71ec8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:36.405483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c64bfa-1a99-443f-9592-277e30f71ec8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:36.411356 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 495/987] 10.222.0.67 () {56 vars in 1122 bytes} [Mon Sep 26 17:45:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:36.416957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7641b85-44f7-40ca-a035-2762f888f316 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:36.432467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:36.526069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7641b85-44f7-40ca-a035-2762f888f316 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:36.561952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95ed237-fe03-437f-a4f0-419c4844d898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:36.563531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ed237-fe03-437f-a4f0-419c4844d898 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:36.563892 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ed237-fe03-437f-a4f0-419c4844d898 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:36.564208 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95ed237-fe03-437f-a4f0-419c4844d898 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:36.564957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ed237-fe03-437f-a4f0-419c4844d898 None admin] RBAC: Adding query filter params (interface=admin, service_id=c0b63082f5ad419db94abd88b5bdfc77, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:36.565307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ed237-fe03-437f-a4f0-419c4844d898 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:36.566753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95ed237-fe03-437f-a4f0-419c4844d898 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:36.576982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 493/988] 10.222.0.67 () {56 vars in 1249 bytes} [Mon Sep 26 17:45:36 2022] GET /identity/v3/endpoints?service_id=c0b63082f5ad419db94abd88b5bdfc77&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:36.579302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c64bfa-1a99-443f-9592-277e30f71ec8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:36.684786 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c64bfa-1a99-443f-9592-277e30f71ec8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:36.702244 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:36.720546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f183fa-ae46-481e-aece-c055ac161839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:36.721164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f183fa-ae46-481e-aece-c055ac161839 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:36.721440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f183fa-ae46-481e-aece-c055ac161839 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:36.721708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f183fa-ae46-481e-aece-c055ac161839 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:36.722303 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f183fa-ae46-481e-aece-c055ac161839 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:36.723680 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f183fa-ae46-481e-aece-c055ac161839 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:36.730020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 496/989] 10.222.0.67 () {56 vars in 1087 bytes} [Mon Sep 26 17:45:36 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:45:37.886117 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045bd04-f97b-44fd-bbf2-4e1f40f6cde7 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:37.886668 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045bd04-f97b-44fd-bbf2-4e1f40f6cde7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:37.886749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0045bd04-f97b-44fd-bbf2-4e1f40f6cde7 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:37.888313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 494/990] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:37.893839 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d181442-6e35-4e50-8817-71fc7f3651d4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:37.894285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d181442-6e35-4e50-8817-71fc7f3651d4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:37.894382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d181442-6e35-4e50-8817-71fc7f3651d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:37.913308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8d181442-6e35-4e50-8817-71fc7f3651d4 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:37.914507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8d181442-6e35-4e50-8817-71fc7f3651d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:37.995692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 497/991] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:37 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:38.188903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973e0ef-ce4b-496d-aa13-d22869bbcff5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.189258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973e0ef-ce4b-496d-aa13-d22869bbcff5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.189441 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973e0ef-ce4b-496d-aa13-d22869bbcff5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.190939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 495/992] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:38.195925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3ef2a-93e3-4f0a-8030-dc83086b29d9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.196212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3ef2a-93e3-4f0a-8030-dc83086b29d9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.196500 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c3ef2a-93e3-4f0a-8030-dc83086b29d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.214656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-04c3ef2a-93e3-4f0a-8030-dc83086b29d9 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:38.215925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-04c3ef2a-93e3-4f0a-8030-dc83086b29d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:38.264285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973e0ef-ce4b-496d-aa13-d22869bbcff5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.284644 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 498/993] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:38 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:38.346548 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:38.370386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973e0ef-ce4b-496d-aa13-d22869bbcff5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:38.407236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf2f73-dfec-450e-afbd-2cfc849e4714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:38.407893 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf2f73-dfec-450e-afbd-2cfc849e4714 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.408174 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf2f73-dfec-450e-afbd-2cfc849e4714 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.408445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bf2f73-dfec-450e-afbd-2cfc849e4714 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.409021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf2f73-dfec-450e-afbd-2cfc849e4714 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:38.410406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bf2f73-dfec-450e-afbd-2cfc849e4714 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-48bf2f73-dfec-450e-afbd-2cfc849e4714 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:38.419258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:38.420232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:38.421634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:38.421634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:38.421634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:38.421634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:38.421634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:38.421634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:38.421634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:38.421634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:38.421634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:38.421634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:38.421634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:38.421634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:38.421634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:45:38.421634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:38.421634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 496/994] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:38.426252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3ef2a-93e3-4f0a-8030-dc83086b29d9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.472225 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:38.529242 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c3ef2a-93e3-4f0a-8030-dc83086b29d9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:38.556086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf2f73-dfec-450e-afbd-2cfc849e4714 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.567203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6971f8-bdf4-4242-bb80-ec7a3f372445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:38.567845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6971f8-bdf4-4242-bb80-ec7a3f372445 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.568148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6971f8-bdf4-4242-bb80-ec7a3f372445 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.568407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6971f8-bdf4-4242-bb80-ec7a3f372445 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.569037 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6971f8-bdf4-4242-bb80-ec7a3f372445 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:38.569508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6971f8-bdf4-4242-bb80-ec7a3f372445 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:38.570711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6971f8-bdf4-4242-bb80-ec7a3f372445 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:38.579729 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 499/995] 10.222.0.67 () {56 vars in 1112 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:38.582854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6971f8-bdf4-4242-bb80-ec7a3f372445 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.662465 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bf2f73-dfec-450e-afbd-2cfc849e4714 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:38.701047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6971f8-bdf4-4242-bb80-ec7a3f372445 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:38.709588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:38.711293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9148353-cf8e-4036-9917-2c258824dbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:38.711980 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9148353-cf8e-4036-9917-2c258824dbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.712260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9148353-cf8e-4036-9917-2c258824dbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.712533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9148353-cf8e-4036-9917-2c258824dbe9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.713123 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9148353-cf8e-4036-9917-2c258824dbe9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:38.714509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9148353-cf8e-4036-9917-2c258824dbe9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c9148353-cf8e-4036-9917-2c258824dbe9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:38.725155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:38.726293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:38.727277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:38.727277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:38.727277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:38.727277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:38.727277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:38.727277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:38.727277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:38.727277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:38.727277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:38.727277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:38.727277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:38.727277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:38.727277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:45:38.727277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:38.727277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 497/996] 10.222.0.67 () {56 vars in 1105 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:38.731925 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9148353-cf8e-4036-9917-2c258824dbe9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.736607 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:38.741200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cd059-ed3f-4dc5-8936-511bfc9dba0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:38.742170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cd059-ed3f-4dc5-8936-511bfc9dba0d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.742904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cd059-ed3f-4dc5-8936-511bfc9dba0d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.743177 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4cd059-ed3f-4dc5-8936-511bfc9dba0d None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.743843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cd059-ed3f-4dc5-8936-511bfc9dba0d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:38.746159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cd059-ed3f-4dc5-8936-511bfc9dba0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:38.746159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef4cd059-ed3f-4dc5-8936-511bfc9dba0d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:38.756969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 500/997] 10.222.0.67 () {56 vars in 1112 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:38.758269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cd059-ed3f-4dc5-8936-511bfc9dba0d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.848794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9148353-cf8e-4036-9917-2c258824dbe9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:38.873292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4cd059-ed3f-4dc5-8936-511bfc9dba0d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:38.886520 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d4b41-8ec1-417e-8046-8fa54a53693b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:38.887145 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d4b41-8ec1-417e-8046-8fa54a53693b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.887415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d4b41-8ec1-417e-8046-8fa54a53693b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.887826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0d4b41-8ec1-417e-8046-8fa54a53693b None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.888863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d4b41-8ec1-417e-8046-8fa54a53693b None admin] RBAC: Adding query filter params (name=image) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:38.889171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d4b41-8ec1-417e-8046-8fa54a53693b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:38.891293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0d4b41-8ec1-417e-8046-8fa54a53693b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:38.896377 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 498/998] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:38.901988 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d4b41-8ec1-417e-8046-8fa54a53693b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.910673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf9bc5-e0e3-40bf-9453-47d324a4baf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:38.911283 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf9bc5-e0e3-40bf-9453-47d324a4baf7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:38.911564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf9bc5-e0e3-40bf-9453-47d324a4baf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:38.911874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf9bc5-e0e3-40bf-9453-47d324a4baf7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:38.912574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf9bc5-e0e3-40bf-9453-47d324a4baf7 None admin] RBAC: Adding query filter params (interface=public, service_id=f05e4e3842234ecaa915b477416b7d99, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:38.912885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf9bc5-e0e3-40bf-9453-47d324a4baf7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:38.914247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf9bc5-e0e3-40bf-9453-47d324a4baf7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:38.919355 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 501/999] 10.222.0.67 () {56 vars in 1251 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/endpoints?service_id=f05e4e3842234ecaa915b477416b7d99&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:38.924941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf9bc5-e0e3-40bf-9453-47d324a4baf7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:38.958280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:38.974552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:39.008606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0d4b41-8ec1-417e-8046-8fa54a53693b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:39.026570 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf9bc5-e0e3-40bf-9453-47d324a4baf7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:39.043137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec5b74-1757-4297-a77a-beac9db30c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:39.043787 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec5b74-1757-4297-a77a-beac9db30c3a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:39.044069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec5b74-1757-4297-a77a-beac9db30c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:39.044344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec5b74-1757-4297-a77a-beac9db30c3a None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:39.044973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec5b74-1757-4297-a77a-beac9db30c3a None admin] RBAC: Adding query filter params (type=image) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:39.045274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec5b74-1757-4297-a77a-beac9db30c3a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:39.046704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec5b74-1757-4297-a77a-beac9db30c3a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:39.051335 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 499/1000] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:39.052540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981da85-61b6-4b90-81f5-ba7163daec4b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:39.052824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981da85-61b6-4b90-81f5-ba7163daec4b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:39.053106 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5981da85-61b6-4b90-81f5-ba7163daec4b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:39.054483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 500/1001] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:39 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:39.060168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5fd2a-6f28-4bfb-a8b1-8d6f8039494a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:39.060450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5fd2a-6f28-4bfb-a8b1-8d6f8039494a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:39.060728 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e5fd2a-6f28-4bfb-a8b1-8d6f8039494a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:39.064034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a623f-cec7-497f-a2c8-0819f43c9879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:39.064990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a623f-cec7-497f-a2c8-0819f43c9879 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:39.065460 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a623f-cec7-497f-a2c8-0819f43c9879 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:39.065919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510a623f-cec7-497f-a2c8-0819f43c9879 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:39.066735 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a623f-cec7-497f-a2c8-0819f43c9879 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:39.068362 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510a623f-cec7-497f-a2c8-0819f43c9879 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:39.075984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 502/1002] 10.222.0.67 () {56 vars in 1087 bytes} [Mon Sep 26 17:45:38 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 17:45:39.081532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-09e5fd2a-6f28-4bfb-a8b1-8d6f8039494a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:39.082815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-09e5fd2a-6f28-4bfb-a8b1-8d6f8039494a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:39.172554 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 501/1003] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:39.446702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a623f-cec7-497f-a2c8-0819f43c9879 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:39.546598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:39.550919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510a623f-cec7-497f-a2c8-0819f43c9879 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:39.586434 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490989b-1acd-431f-95c2-656508e17775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:39.587078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490989b-1acd-431f-95c2-656508e17775 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:39.587363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490989b-1acd-431f-95c2-656508e17775 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:39.587690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5490989b-1acd-431f-95c2-656508e17775 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:39.588250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490989b-1acd-431f-95c2-656508e17775 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:39.589720 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5490989b-1acd-431f-95c2-656508e17775 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5490989b-1acd-431f-95c2-656508e17775 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:39.599706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:39.600679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:39.601484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:39.601484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:39.601484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:39.601484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:39.601484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:39.601484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:39.601484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:39.601484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:39.601484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:39.601484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:39.601484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:39.601484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:39.601484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 26 17:45:39.601484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:39.603962 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 503/1004] 10.222.0.67 () {56 vars in 1126 bytes} [Mon Sep 26 17:45:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:39.606651 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5fd2a-6f28-4bfb-a8b1-8d6f8039494a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:39.673527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:39.714538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e5fd2a-6f28-4bfb-a8b1-8d6f8039494a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:39.749205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388365c5-1950-434e-aa7b-b68ecbcb9be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:39.749927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388365c5-1950-434e-aa7b-b68ecbcb9be3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:39.750215 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388365c5-1950-434e-aa7b-b68ecbcb9be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:39.750485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388365c5-1950-434e-aa7b-b68ecbcb9be3 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:39.751113 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388365c5-1950-434e-aa7b-b68ecbcb9be3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:39.751419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388365c5-1950-434e-aa7b-b68ecbcb9be3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:39.752843 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388365c5-1950-434e-aa7b-b68ecbcb9be3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:39.762330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 502/1005] 10.222.0.67 () {56 vars in 1122 bytes} [Mon Sep 26 17:45:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:39.763768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490989b-1acd-431f-95c2-656508e17775 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:39.868066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5490989b-1acd-431f-95c2-656508e17775 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:39.902225 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a34eab-97dd-4ee0-b70b-967e4a977d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:39.902836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a34eab-97dd-4ee0-b70b-967e4a977d78 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:39.903119 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a34eab-97dd-4ee0-b70b-967e4a977d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:39.903488 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a34eab-97dd-4ee0-b70b-967e4a977d78 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:39.904044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a34eab-97dd-4ee0-b70b-967e4a977d78 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:39.904347 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a34eab-97dd-4ee0-b70b-967e4a977d78 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:39.906284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a34eab-97dd-4ee0-b70b-967e4a977d78 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:39.915848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 504/1006] 10.222.0.67 () {56 vars in 1122 bytes} [Mon Sep 26 17:45:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:39.916777 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388365c5-1950-434e-aa7b-b68ecbcb9be3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:40.030081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388365c5-1950-434e-aa7b-b68ecbcb9be3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:40.060556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:40.069814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a13eb77-a65a-41e0-9efb-5c73dbba1150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:40.070418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13eb77-a65a-41e0-9efb-5c73dbba1150 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:40.070693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13eb77-a65a-41e0-9efb-5c73dbba1150 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:40.078407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a13eb77-a65a-41e0-9efb-5c73dbba1150 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:40.078407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13eb77-a65a-41e0-9efb-5c73dbba1150 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:40.078407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a13eb77-a65a-41e0-9efb-5c73dbba1150 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:40.166386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 503/1007] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 17:45:39 2022] POST /identity/v3/endpoints => generated 325 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:45:41.248815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676ea59-10d2-40c0-9813-bf7d996a9e82 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:41.249326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676ea59-10d2-40c0-9813-bf7d996a9e82 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:41.249326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676ea59-10d2-40c0-9813-bf7d996a9e82 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:41.251203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 505/1008] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:41.257494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ef5d0-3c8c-44af-89b1-5d686d02a7f6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:41.258009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ef5d0-3c8c-44af-89b1-5d686d02a7f6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:41.258009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ef5d0-3c8c-44af-89b1-5d686d02a7f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:41.277855 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3e1ef5d0-3c8c-44af-89b1-5d686d02a7f6 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:41.279077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3e1ef5d0-3c8c-44af-89b1-5d686d02a7f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:41.353863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 504/1009] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:41 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:41.378176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0fbc2-35cf-4350-b1e1-7891a6ff4a84 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:41.378491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0fbc2-35cf-4350-b1e1-7891a6ff4a84 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:41.378734 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f0fbc2-35cf-4350-b1e1-7891a6ff4a84 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:41.380394 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 506/1010] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:41.385287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611227f-039d-45ee-abf4-86495da84734 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:41.385583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611227f-039d-45ee-abf4-86495da84734 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:41.385864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7611227f-039d-45ee-abf4-86495da84734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:41.404623 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7611227f-039d-45ee-abf4-86495da84734 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:41.406148 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7611227f-039d-45ee-abf4-86495da84734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:41.476828 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 505/1011] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:41 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:41.633586 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0fbc2-35cf-4350-b1e1-7891a6ff4a84 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:41.666930 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:41.747242 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f0fbc2-35cf-4350-b1e1-7891a6ff4a84 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:41.770966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611227f-039d-45ee-abf4-86495da84734 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:41.785627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b820ec-4f0e-457e-a7fb-9e68e66f78e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:41.786432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b820ec-4f0e-457e-a7fb-9e68e66f78e6 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:41.786713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b820ec-4f0e-457e-a7fb-9e68e66f78e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:41.787379 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b820ec-4f0e-457e-a7fb-9e68e66f78e6 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:41.788070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b820ec-4f0e-457e-a7fb-9e68e66f78e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:41.789429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b820ec-4f0e-457e-a7fb-9e68e66f78e6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:41.817236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:41.863021 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 507/1012] 10.222.0.67 () {60 vars in 1141 bytes} [Mon Sep 26 17:45:41 2022] POST /identity/v3/services => generated 242 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:45:41.880461 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7611227f-039d-45ee-abf4-86495da84734 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:41.930178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fca03-b6d1-4cc0-90a0-08bdd4200c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:41.930740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fca03-b6d1-4cc0-90a0-08bdd4200c4e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:41.931020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fca03-b6d1-4cc0-90a0-08bdd4200c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:41.931287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73fca03-b6d1-4cc0-90a0-08bdd4200c4e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:41.931907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fca03-b6d1-4cc0-90a0-08bdd4200c4e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:41.933300 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73fca03-b6d1-4cc0-90a0-08bdd4200c4e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d73fca03-b6d1-4cc0-90a0-08bdd4200c4e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:41.943481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:41.944584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:41.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:41.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:41.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:41.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:41.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:41.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:41.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:41.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:41.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:41.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:41.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:41.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:41.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 26 17:45:41.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:41.945399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 506/1013] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:41.950842 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b820ec-4f0e-457e-a7fb-9e68e66f78e6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:42.053531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:42.069658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b820ec-4f0e-457e-a7fb-9e68e66f78e6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:42.105212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6b0bf-07b4-42d7-a1cd-97b4ee520af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:42.106096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6b0bf-07b4-42d7-a1cd-97b4ee520af1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:42.106529 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6b0bf-07b4-42d7-a1cd-97b4ee520af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:42.107126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6b0bf-07b4-42d7-a1cd-97b4ee520af1 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:42.107927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6b0bf-07b4-42d7-a1cd-97b4ee520af1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:42.108399 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6b0bf-07b4-42d7-a1cd-97b4ee520af1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:42.110803 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6b0bf-07b4-42d7-a1cd-97b4ee520af1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:42.115922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 508/1014] 10.222.0.67 () {56 vars in 1112 bytes} [Mon Sep 26 17:45:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:42.121881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fca03-b6d1-4cc0-90a0-08bdd4200c4e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:42.227016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73fca03-b6d1-4cc0-90a0-08bdd4200c4e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:42.228528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:42.265569 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b8b91-2157-4152-9c96-d6abff367feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:42.267108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b8b91-2157-4152-9c96-d6abff367feb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:42.267544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b8b91-2157-4152-9c96-d6abff367feb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:42.267995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b8b91-2157-4152-9c96-d6abff367feb None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:42.268827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b8b91-2157-4152-9c96-d6abff367feb None admin] RBAC: Adding query filter params (type=compute) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:42.269290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b8b91-2157-4152-9c96-d6abff367feb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:42.270878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b8b91-2157-4152-9c96-d6abff367feb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:42.277408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 507/1015] 10.222.0.67 () {56 vars in 1112 bytes} [Mon Sep 26 17:45:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:42.283725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6b0bf-07b4-42d7-a1cd-97b4ee520af1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:42.376095 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2001aa-3f98-4eb2-a5de-7c7912d95a3d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:42.376482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2001aa-3f98-4eb2-a5de-7c7912d95a3d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:42.376763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2001aa-3f98-4eb2-a5de-7c7912d95a3d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:42.380124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 508/1016] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:42.383343 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0252e-82f6-49f3-b306-c8af3d9e56fb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:42.383628 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0252e-82f6-49f3-b306-c8af3d9e56fb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:42.383933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0252e-82f6-49f3-b306-c8af3d9e56fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:42.394774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6b0bf-07b4-42d7-a1cd-97b4ee520af1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:42.403073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c7a0252e-82f6-49f3-b306-c8af3d9e56fb None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:42.404282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c7a0252e-82f6-49f3-b306-c8af3d9e56fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:42.437382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd1f92-7835-4c17-9ff2-ff31fd6ac399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:42.438004 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd1f92-7835-4c17-9ff2-ff31fd6ac399 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:42.438286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd1f92-7835-4c17-9ff2-ff31fd6ac399 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:42.438562 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd1f92-7835-4c17-9ff2-ff31fd6ac399 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:42.439226 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd1f92-7835-4c17-9ff2-ff31fd6ac399 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:42.441802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cd1f92-7835-4c17-9ff2-ff31fd6ac399 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:42.473319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 509/1017] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:42 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:42.513701 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 509/1018] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 17:45:42 2022] POST /identity/v3/endpoints => generated 334 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:45:42.749497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0252e-82f6-49f3-b306-c8af3d9e56fb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:42.856673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:42.862998 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0252e-82f6-49f3-b306-c8af3d9e56fb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:42.904535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159cf31-14a8-40e5-82b7-6cc1b84a6bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:42.905181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159cf31-14a8-40e5-82b7-6cc1b84a6bf0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:42.906247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159cf31-14a8-40e5-82b7-6cc1b84a6bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:42.906597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159cf31-14a8-40e5-82b7-6cc1b84a6bf0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:42.908076 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159cf31-14a8-40e5-82b7-6cc1b84a6bf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:42.909842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159cf31-14a8-40e5-82b7-6cc1b84a6bf0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:42.911609 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 510/1019] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:42.917177 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd1f92-7835-4c17-9ff2-ff31fd6ac399 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:42.926662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:43.028969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd1f92-7835-4c17-9ff2-ff31fd6ac399 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:43.063612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb239df2-5b2f-44ae-b577-1d6396b2efea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:43.064248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb239df2-5b2f-44ae-b577-1d6396b2efea None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:43.064525 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb239df2-5b2f-44ae-b577-1d6396b2efea None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:43.064791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb239df2-5b2f-44ae-b577-1d6396b2efea None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:43.065428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb239df2-5b2f-44ae-b577-1d6396b2efea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:43.066843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb239df2-5b2f-44ae-b577-1d6396b2efea None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:43.142861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 510/1020] 10.222.0.67 () {60 vars in 1141 bytes} [Mon Sep 26 17:45:42 2022] POST /identity/v3/projects => generated 311 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:45:44.106077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3466f-4838-4c69-a825-7e0fb0e50a6e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:44.106610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3466f-4838-4c69-a825-7e0fb0e50a6e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:44.106610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c3466f-4838-4c69-a825-7e0fb0e50a6e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:44.108042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 511/1021] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:44.113791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf1965-b07a-4cc6-bd1d-7e15ae7b05f7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:44.114212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf1965-b07a-4cc6-bd1d-7e15ae7b05f7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:44.114397 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf1965-b07a-4cc6-bd1d-7e15ae7b05f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:44.133083 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-22cf1965-b07a-4cc6-bd1d-7e15ae7b05f7 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:44.134271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-22cf1965-b07a-4cc6-bd1d-7e15ae7b05f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:44.188707 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:44.201963 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 511/1022] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:44.470837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3466f-4838-4c69-a825-7e0fb0e50a6e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:44.579945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c3466f-4838-4c69-a825-7e0fb0e50a6e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:44.615938 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93466c07-a631-4754-96c0-3d083229bc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:44.616569 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93466c07-a631-4754-96c0-3d083229bc6e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:44.616847 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93466c07-a631-4754-96c0-3d083229bc6e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:44.617114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93466c07-a631-4754-96c0-3d083229bc6e None admin] PATH_INFO: `/v3/services/image` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:44.617685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93466c07-a631-4754-96c0-3d083229bc6e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:44.619049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93466c07-a631-4754-96c0-3d083229bc6e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-93466c07-a631-4754-96c0-3d083229bc6e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:44.628882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:44.629966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:44.632784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:44.632784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:44.632784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:44.632784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:44.632784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:44.632784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:44.632784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:44.632784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:44.632784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:44.632784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:44.632784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:44.632784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:44.632784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:45:44.632784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:44.632784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 512/1023] 10.222.0.67 () {56 vars in 1105 bytes} [Mon Sep 26 17:45:44 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:44.636099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf1965-b07a-4cc6-bd1d-7e15ae7b05f7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:44.741472 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf1965-b07a-4cc6-bd1d-7e15ae7b05f7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:44.778482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d07ae-ae05-44ed-be71-1cfe3e6df7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:44.779102 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d07ae-ae05-44ed-be71-1cfe3e6df7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:44.779387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d07ae-ae05-44ed-be71-1cfe3e6df7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:44.779686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d07ae-ae05-44ed-be71-1cfe3e6df7f6 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:44.780848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d07ae-ae05-44ed-be71-1cfe3e6df7f6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:44.781245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d07ae-ae05-44ed-be71-1cfe3e6df7f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:44.782624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63d07ae-ae05-44ed-be71-1cfe3e6df7f6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:44.791866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 512/1024] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:44 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:44.793668 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93466c07-a631-4754-96c0-3d083229bc6e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:44.821508 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:44.852394 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea28223-0712-406c-a0de-e0297f060f9f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:44.852745 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea28223-0712-406c-a0de-e0297f060f9f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:44.853361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea28223-0712-406c-a0de-e0297f060f9f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:44.854709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 513/1025] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:44.860317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37957e-8494-46a7-ba31-1b9555a31b26 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:44.860882 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37957e-8494-46a7-ba31-1b9555a31b26 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:44.861436 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37957e-8494-46a7-ba31-1b9555a31b26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:44.882186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-be37957e-8494-46a7-ba31-1b9555a31b26 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:44.883794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-be37957e-8494-46a7-ba31-1b9555a31b26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:44.921545 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93466c07-a631-4754-96c0-3d083229bc6e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:44.935222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:44.957723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656532f6-866f-4790-9c42-bba5d401d213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:44.958707 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656532f6-866f-4790-9c42-bba5d401d213 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:44.958988 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656532f6-866f-4790-9c42-bba5d401d213 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:44.959262 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656532f6-866f-4790-9c42-bba5d401d213 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:44.959929 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656532f6-866f-4790-9c42-bba5d401d213 None admin] RBAC: Adding query filter params (type=image) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:44.960236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656532f6-866f-4790-9c42-bba5d401d213 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:44.961604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656532f6-866f-4790-9c42-bba5d401d213 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:44.963938 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 514/1026] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:44 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:44.967103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 513/1027] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:44 2022] GET /identity/v3/services?type=image => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:44.973453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37957e-8494-46a7-ba31-1b9555a31b26 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:45.083180 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37957e-8494-46a7-ba31-1b9555a31b26 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:45.117147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7a690-afa3-45e8-9a81-cdfebb1b8249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:45.117782 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7a690-afa3-45e8-9a81-cdfebb1b8249 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:45.118022 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7a690-afa3-45e8-9a81-cdfebb1b8249 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:45.118293 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7a690-afa3-45e8-9a81-cdfebb1b8249 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:45.118988 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7a690-afa3-45e8-9a81-cdfebb1b8249 None admin] RBAC: Adding query filter params (interface=public, service_id=57d64c35d0524742a1e2c74b165820a5, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:45.119295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7a690-afa3-45e8-9a81-cdfebb1b8249 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:45.120683 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7a690-afa3-45e8-9a81-cdfebb1b8249 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:45.130241 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 515/1028] 10.222.0.67 () {56 vars in 1251 bytes} [Mon Sep 26 17:45:44 2022] GET /identity/v3/endpoints?service_id=57d64c35d0524742a1e2c74b165820a5&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:45.131510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656532f6-866f-4790-9c42-bba5d401d213 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:45.226326 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:45.229407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7a690-afa3-45e8-9a81-cdfebb1b8249 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:45.244108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656532f6-866f-4790-9c42-bba5d401d213 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:45.283615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:45.288170 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6b59c3-e795-4325-9275-982e0792e4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:45.288788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b59c3-e795-4325-9275-982e0792e4bf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:45.289069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b59c3-e795-4325-9275-982e0792e4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:45.289340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6b59c3-e795-4325-9275-982e0792e4bf None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:45.290362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b59c3-e795-4325-9275-982e0792e4bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:45.292073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6b59c3-e795-4325-9275-982e0792e4bf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:45.312196 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 514/1029] 10.222.0.67 () {56 vars in 1087 bytes} [Mon Sep 26 17:45:45 2022] GET /identity/v3/services => generated 1957 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:45:45.344762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7a690-afa3-45e8-9a81-cdfebb1b8249 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:45.356705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fc2e2-27cc-4756-925c-0ad5b2540ad5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:45.357036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fc2e2-27cc-4756-925c-0ad5b2540ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:45.357262 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460fc2e2-27cc-4756-925c-0ad5b2540ad5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:45.359695 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 515/1030] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:45.365442 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb03d8-2e11-4029-9e4b-87a04781dc05 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:45.365724 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb03d8-2e11-4029-9e4b-87a04781dc05 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:45.366246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfb03d8-2e11-4029-9e4b-87a04781dc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:45.384787 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315d5c6-c7d1-4bcc-8814-fcae1758cbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:45.385438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315d5c6-c7d1-4bcc-8814-fcae1758cbed None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:45.385719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315d5c6-c7d1-4bcc-8814-fcae1758cbed None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:45.385993 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c315d5c6-c7d1-4bcc-8814-fcae1758cbed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:45.386121 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-dbfb03d8-2e11-4029-9e4b-87a04781dc05 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:45.387545 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dbfb03d8-2e11-4029-9e4b-87a04781dc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:45.398770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c315d5c6-c7d1-4bcc-8814-fcae1758cbed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c315d5c6-c7d1-4bcc-8814-fcae1758cbed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:45.403869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:45.404847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:45.406003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:45.406003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:45.406003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:45.406003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:45.406003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:45.406003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:45.406003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:45.406003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:45.406003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:45.406003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:45.406003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:45:45.406003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:45.406003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 516/1031] 10.222.0.67 () {56 vars in 1120 bytes} [Mon Sep 26 17:45:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:45.411711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315d5c6-c7d1-4bcc-8814-fcae1758cbed None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:45.471722 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 516/1032] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:45.527572 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c315d5c6-c7d1-4bcc-8814-fcae1758cbed None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:45.536710 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:45.563976 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799beb5-6c21-40b1-9f0b-1569055d4b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:45.565047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799beb5-6c21-40b1-9f0b-1569055d4b00 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:45.565811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799beb5-6c21-40b1-9f0b-1569055d4b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:45.566127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4799beb5-6c21-40b1-9f0b-1569055d4b00 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:45.566795 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799beb5-6c21-40b1-9f0b-1569055d4b00 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:45.567103 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799beb5-6c21-40b1-9f0b-1569055d4b00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:45.568505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4799beb5-6c21-40b1-9f0b-1569055d4b00 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:45.576964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 517/1033] 10.222.0.67 () {56 vars in 1115 bytes} [Mon Sep 26 17:45:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:45.583281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb03d8-2e11-4029-9e4b-87a04781dc05 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:45.667845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:45.686345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfb03d8-2e11-4029-9e4b-87a04781dc05 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:45.721434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e9356-fe6c-419e-8971-e7111c833c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:45.722061 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e9356-fe6c-419e-8971-e7111c833c66 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:45.722341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e9356-fe6c-419e-8971-e7111c833c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:45.722613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392e9356-fe6c-419e-8971-e7111c833c66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:45.732631 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e9356-fe6c-419e-8971-e7111c833c66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:45.734755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392e9356-fe6c-419e-8971-e7111c833c66 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:45.743032 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799beb5-6c21-40b1-9f0b-1569055d4b00 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-392e9356-fe6c-419e-8971-e7111c833c66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:45.751678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:45.752647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:45.753509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:45.753509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:45.753509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:45.753509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 517/1034] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:45.759486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e9356-fe6c-419e-8971-e7111c833c66 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:45.857507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4799beb5-6c21-40b1-9f0b-1569055d4b00 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:45.868810 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392e9356-fe6c-419e-8971-e7111c833c66 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:45.897287 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e675eb5-805e-4f79-a4c5-bf0945010851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:45.898024 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e675eb5-805e-4f79-a4c5-bf0945010851 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:45.898304 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e675eb5-805e-4f79-a4c5-bf0945010851 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:45.898585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e675eb5-805e-4f79-a4c5-bf0945010851 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:45.899972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e675eb5-805e-4f79-a4c5-bf0945010851 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:45.902163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e675eb5-805e-4f79-a4c5-bf0945010851 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:45.904400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 518/1035] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:45.910227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e675eb5-805e-4f79-a4c5-bf0945010851 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:45.912125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8038236-1fbe-4a6d-9531-86d4f5af623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:45.912743 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8038236-1fbe-4a6d-9531-86d4f5af623a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:45.913024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8038236-1fbe-4a6d-9531-86d4f5af623a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:45.913298 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8038236-1fbe-4a6d-9531-86d4f5af623a None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:45.914036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8038236-1fbe-4a6d-9531-86d4f5af623a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:45.914348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8038236-1fbe-4a6d-9531-86d4f5af623a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:45.915752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8038236-1fbe-4a6d-9531-86d4f5af623a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:45.917546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:45.931127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 518/1036] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:45.934362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8038236-1fbe-4a6d-9531-86d4f5af623a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:45.951464 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:46.029291 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e675eb5-805e-4f79-a4c5-bf0945010851 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:46.044496 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8038236-1fbe-4a6d-9531-86d4f5af623a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:46.068999 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8e79e-84de-4a75-aff4-9cc8bca27dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:46.069609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8e79e-84de-4a75-aff4-9cc8bca27dc3 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:46.069889 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8e79e-84de-4a75-aff4-9cc8bca27dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:46.070161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c8e79e-84de-4a75-aff4-9cc8bca27dc3 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:46.070810 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8e79e-84de-4a75-aff4-9cc8bca27dc3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:46.072286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c8e79e-84de-4a75-aff4-9cc8bca27dc3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:46.080988 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00204c5e-2b01-49e2-9c65-d322d6f328ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:46.081722 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00204c5e-2b01-49e2-9c65-d322d6f328ea None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:46.081977 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00204c5e-2b01-49e2-9c65-d322d6f328ea None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:46.082206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00204c5e-2b01-49e2-9c65-d322d6f328ea None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:46.089812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00204c5e-2b01-49e2-9c65-d322d6f328ea None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:46.091366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00204c5e-2b01-49e2-9c65-d322d6f328ea None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-00204c5e-2b01-49e2-9c65-d322d6f328ea None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:46.103544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:46.106499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:46.107851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:46.107851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:46.107851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:46.107851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:46.107851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:46.107851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:46.107851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:46.107851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:46.107851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:45:46.107851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:46.107851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 519/1037] 10.222.0.67 () {56 vars in 1093 bytes} [Mon Sep 26 17:45:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:46.111355 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00204c5e-2b01-49e2-9c65-d322d6f328ea None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:46.151676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 519/1038] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:45:45 2022] POST /identity/v3/users => generated 285 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:45:46.182583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:46.237596 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00204c5e-2b01-49e2-9c65-d322d6f328ea None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:46.274212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69609188-ce76-449b-a0db-fcbda6468601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:46.274842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69609188-ce76-449b-a0db-fcbda6468601 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:46.275120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69609188-ce76-449b-a0db-fcbda6468601 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:46.275394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69609188-ce76-449b-a0db-fcbda6468601 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:46.276169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69609188-ce76-449b-a0db-fcbda6468601 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:46.276546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69609188-ce76-449b-a0db-fcbda6468601 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:46.278050 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69609188-ce76-449b-a0db-fcbda6468601 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:46.303501 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 520/1039] 10.222.0.67 () {56 vars in 1133 bytes} [Mon Sep 26 17:45:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:46.305059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8e79e-84de-4a75-aff4-9cc8bca27dc3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:46.401244 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:46.410566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c8e79e-84de-4a75-aff4-9cc8bca27dc3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:46.451770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f68213-903e-4a7e-85f2-360f853139fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:46.451770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f68213-903e-4a7e-85f2-360f853139fa None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:46.451770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f68213-903e-4a7e-85f2-360f853139fa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:46.451770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f68213-903e-4a7e-85f2-360f853139fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:46.517620 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f68213-903e-4a7e-85f2-360f853139fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:46.519069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f68213-903e-4a7e-85f2-360f853139fa None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:46.610411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:46.620419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 520/1040] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:46 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 17:45:46.626198 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69609188-ce76-449b-a0db-fcbda6468601 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:46.729485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69609188-ce76-449b-a0db-fcbda6468601 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:46.742655 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:46.764195 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00897bc-22e7-422f-9169-e0e9c263a9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:46.765369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00897bc-22e7-422f-9169-e0e9c263a9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:46.765651 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00897bc-22e7-422f-9169-e0e9c263a9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:46.765928 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00897bc-22e7-422f-9169-e0e9c263a9f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:46.776792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00897bc-22e7-422f-9169-e0e9c263a9f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:46.778249 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00897bc-22e7-422f-9169-e0e9c263a9f7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d00897bc-22e7-422f-9169-e0e9c263a9f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:46.792879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:46.793815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:46.794721 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:46.794721 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:46.794721 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:46.796041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 521/1041] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:46.799737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f68213-903e-4a7e-85f2-360f853139fa None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:46.900138 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f68213-903e-4a7e-85f2-360f853139fa None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:46.934093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d58d4c-1391-4e09-a905-0d2502de0c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:46.934715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d58d4c-1391-4e09-a905-0d2502de0c34 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:46.935002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d58d4c-1391-4e09-a905-0d2502de0c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:46.935278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d58d4c-1391-4e09-a905-0d2502de0c34 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:46.935941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d58d4c-1391-4e09-a905-0d2502de0c34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:46.936245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d58d4c-1391-4e09-a905-0d2502de0c34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:46.938425 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d58d4c-1391-4e09-a905-0d2502de0c34 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:46.950268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 521/1042] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:46.956291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00897bc-22e7-422f-9169-e0e9c263a9f7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:47.069784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00897bc-22e7-422f-9169-e0e9c263a9f7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:47.105201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f3674-f9c8-4bac-aa9b-3068e47f01f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:47.105865 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f3674-f9c8-4bac-aa9b-3068e47f01f0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:47.106147 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f3674-f9c8-4bac-aa9b-3068e47f01f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:47.106422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f3674-f9c8-4bac-aa9b-3068e47f01f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:47.114864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f3674-f9c8-4bac-aa9b-3068e47f01f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:47.116371 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f3674-f9c8-4bac-aa9b-3068e47f01f0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:47.128041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-297f3674-f9c8-4bac-aa9b-3068e47f01f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:47.128041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:47.128041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:47.128041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:47.128041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:47.128041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:47.128041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:47.128041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:47.128041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:47.128041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:47.128041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:47.128041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:47.128041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:47.128041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:47.128041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:47.128927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:47.129825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:47.129825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:47.129825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:47.129825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:47.129825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:47.129825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:47.129825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:47.129825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:47.130831 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 522/1043] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:47.134993 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d58d4c-1391-4e09-a905-0d2502de0c34 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:47.191156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:47.240027 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d58d4c-1391-4e09-a905-0d2502de0c34 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:47.274650 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586d5a4-bb38-47cf-a4dd-e393abe1d9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:47.275264 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586d5a4-bb38-47cf-a4dd-e393abe1d9f7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:47.275546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586d5a4-bb38-47cf-a4dd-e393abe1d9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:47.275851 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586d5a4-bb38-47cf-a4dd-e393abe1d9f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:47.276536 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586d5a4-bb38-47cf-a4dd-e393abe1d9f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:47.276840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586d5a4-bb38-47cf-a4dd-e393abe1d9f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:47.278265 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586d5a4-bb38-47cf-a4dd-e393abe1d9f7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:47.295303 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 522/1044] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:47.297600 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f3674-f9c8-4bac-aa9b-3068e47f01f0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:47.325067 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:47.403012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f3674-f9c8-4bac-aa9b-3068e47f01f0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:47.445131 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384bc28a-750b-4e09-9b20-ab4a18373ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:47.445761 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bc28a-750b-4e09-9b20-ab4a18373ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:47.446036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bc28a-750b-4e09-9b20-ab4a18373ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:47.446305 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384bc28a-750b-4e09-9b20-ab4a18373ff2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:47.447043 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384bc28a-750b-4e09-9b20-ab4a18373ff2 None admin] RBAC: Adding query filter params (role.id=fb6eaf230a474167b3a3726cbd2d35aa, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=b9ed29a3f6be4d64bb3751f9e95eb56b) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:47.447350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384bc28a-750b-4e09-9b20-ab4a18373ff2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:47.448860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384bc28a-750b-4e09-9b20-ab4a18373ff2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:47.460823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 523/1045] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:47 2022] GET /identity/v3/role_assignments?user.id=b9ed29a3f6be4d64bb3751f9e95eb56b&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=fb6eaf230a474167b3a3726cbd2d35aa => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:47.553352 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d60f9-bce7-4122-984b-5f3688a1164a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:47.553544 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d60f9-bce7-4122-984b-5f3688a1164a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:47.553804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63d60f9-bce7-4122-984b-5f3688a1164a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:47.555322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 523/1046] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:47.560811 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e994c-1327-4590-afb3-97ee52f79d80 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:47.561092 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e994c-1327-4590-afb3-97ee52f79d80 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:47.561370 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1e994c-1327-4590-afb3-97ee52f79d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:47.582043 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8a1e994c-1327-4590-afb3-97ee52f79d80 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:47.583264 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8a1e994c-1327-4590-afb3-97ee52f79d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:47.656794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 524/1047] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:47.921718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d60f9-bce7-4122-984b-5f3688a1164a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.031119 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63d60f9-bce7-4122-984b-5f3688a1164a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:48.045241 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:48.067555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37331dc-ee3b-4683-9ea6-3cd00e846774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:48.069192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37331dc-ee3b-4683-9ea6-3cd00e846774 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.069192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37331dc-ee3b-4683-9ea6-3cd00e846774 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.069192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37331dc-ee3b-4683-9ea6-3cd00e846774 None admin] PATH_INFO: `/v3/services/image` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.069538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37331dc-ee3b-4683-9ea6-3cd00e846774 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:48.070800 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37331dc-ee3b-4683-9ea6-3cd00e846774 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-d37331dc-ee3b-4683-9ea6-3cd00e846774 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:48.079532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:48.080643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:48.081665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:48.081665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:48.081665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:48.081665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:48.081665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:48.081665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:48.081665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:48.081665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:48.081665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:48.081665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:48.081665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:45:48.081665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:45:48.081665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 26 17:45:48.081665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:48.081665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 524/1048] 10.222.0.67 () {56 vars in 1105 bytes} [Mon Sep 26 17:45:47 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:48.088532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e994c-1327-4590-afb3-97ee52f79d80 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.107571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:48.192664 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1e994c-1327-4590-afb3-97ee52f79d80 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:48.227507 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0792c4-f878-4674-bc6d-6d4c796b9342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:48.228141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0792c4-f878-4674-bc6d-6d4c796b9342 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.228417 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0792c4-f878-4674-bc6d-6d4c796b9342 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.228686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0792c4-f878-4674-bc6d-6d4c796b9342 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.229304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0792c4-f878-4674-bc6d-6d4c796b9342 None admin] RBAC: Adding query filter params (name=image) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:48.229599 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0792c4-f878-4674-bc6d-6d4c796b9342 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:48.230950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0792c4-f878-4674-bc6d-6d4c796b9342 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:48.241022 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 525/1049] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:48.242244 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37331dc-ee3b-4683-9ea6-3cd00e846774 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.349343 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37331dc-ee3b-4683-9ea6-3cd00e846774 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:48.385181 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b1f657-eb41-4b6d-938c-729f07afed5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:48.385802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1f657-eb41-4b6d-938c-729f07afed5f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.386078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1f657-eb41-4b6d-938c-729f07afed5f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.386412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b1f657-eb41-4b6d-938c-729f07afed5f None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.387049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1f657-eb41-4b6d-938c-729f07afed5f None admin] RBAC: Adding query filter params (type=image) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:48.387345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1f657-eb41-4b6d-938c-729f07afed5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:48.389089 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b1f657-eb41-4b6d-938c-729f07afed5f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:48.399791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 525/1050] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:48.401038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0792c4-f878-4674-bc6d-6d4c796b9342 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.441260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5efd3-0cd5-4066-b79e-155957a95ee4 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.441551 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5efd3-0cd5-4066-b79e-155957a95ee4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.441828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5efd3-0cd5-4066-b79e-155957a95ee4 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.443334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 526/1051] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:48.447982 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee19a-4d76-4712-bab5-38ef7dacde54 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.448489 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee19a-4d76-4712-bab5-38ef7dacde54 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.448597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ee19a-4d76-4712-bab5-38ef7dacde54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.467372 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-701ee19a-4d76-4712-bab5-38ef7dacde54 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:48.469013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-701ee19a-4d76-4712-bab5-38ef7dacde54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:48.493329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:48.504896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:48.513783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0792c4-f878-4674-bc6d-6d4c796b9342 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:48.542835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 527/1052] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:48.556514 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0658f-67f8-464d-8b26-46ea8f10dbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:48.557265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0658f-67f8-464d-8b26-46ea8f10dbd2 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.557750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0658f-67f8-464d-8b26-46ea8f10dbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.558095 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0658f-67f8-464d-8b26-46ea8f10dbd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:48.558829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0658f-67f8-464d-8b26-46ea8f10dbd2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:48.560307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0658f-67f8-464d-8b26-46ea8f10dbd2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:48.667710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 526/1053] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 17:45:48 2022] POST /identity/v3/endpoints => generated 327 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 17:45:48.834172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee19a-4d76-4712-bab5-38ef7dacde54 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:48.944935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ee19a-4d76-4712-bab5-38ef7dacde54 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:48.994780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f654e-e2b7-4e33-9706-718c1ac6a62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:48.995423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f654e-e2b7-4e33-9706-718c1ac6a62f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:48.995736 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f654e-e2b7-4e33-9706-718c1ac6a62f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:48.996007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f654e-e2b7-4e33-9706-718c1ac6a62f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.007326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f654e-e2b7-4e33-9706-718c1ac6a62f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-607f654e-e2b7-4e33-9706-718c1ac6a62f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:49.014280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:49.015980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:49.016823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:49.016823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:49.016823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:49.016823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:49.016823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:49.016823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:49.016823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:49.016823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:49.016823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:49.016823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:49.016823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:49.016823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:49.016823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 528/1054] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:49.024627 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0658f-67f8-464d-8b26-46ea8f10dbd2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:49.175823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0658f-67f8-464d-8b26-46ea8f10dbd2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:49.224015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a83483-2d5b-4f48-8143-8ac51c9853d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:49.224643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83483-2d5b-4f48-8143-8ac51c9853d4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.224923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83483-2d5b-4f48-8143-8ac51c9853d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.225194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a83483-2d5b-4f48-8143-8ac51c9853d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.225862 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a83483-2d5b-4f48-8143-8ac51c9853d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:49.226161 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a83483-2d5b-4f48-8143-8ac51c9853d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:49.227530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a83483-2d5b-4f48-8143-8ac51c9853d4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:49.235577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:49.245346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 527/1055] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:49.246861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f654e-e2b7-4e33-9706-718c1ac6a62f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:49.287466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:49.400813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f654e-e2b7-4e33-9706-718c1ac6a62f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:49.445764 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf935f98-682e-47f7-a6b4-c9b29cc88936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:49.446322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf935f98-682e-47f7-a6b4-c9b29cc88936 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.446598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf935f98-682e-47f7-a6b4-c9b29cc88936 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.446865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf935f98-682e-47f7-a6b4-c9b29cc88936 None admin] PATH_INFO: `/v3/users/59e68dadce234da3bcf75dcbce5582be` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.467096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf935f98-682e-47f7-a6b4-c9b29cc88936 None admin] RBAC: Authorizing `identity:get_user(user_id=59e68dadce234da3bcf75dcbce5582be)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:49.470208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf935f98-682e-47f7-a6b4-c9b29cc88936 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:49.472101 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 529/1056] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:49 2022] GET /identity/v3/users/59e68dadce234da3bcf75dcbce5582be => generated 285 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:49.478016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a83483-2d5b-4f48-8143-8ac51c9853d4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:49.613118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a83483-2d5b-4f48-8143-8ac51c9853d4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:49.654243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe700d-6d81-46ea-a38f-70f1765193dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:49.654870 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe700d-6d81-46ea-a38f-70f1765193dd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.655747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe700d-6d81-46ea-a38f-70f1765193dd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.656174 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fe700d-6d81-46ea-a38f-70f1765193dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.755766 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:49.759839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe700d-6d81-46ea-a38f-70f1765193dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:49.761358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fe700d-6d81-46ea-a38f-70f1765193dd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:49.824048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d48dc4-ec7c-40e3-a3a3-52aab879ae2a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.824462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d48dc4-ec7c-40e3-a3a3-52aab879ae2a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.824901 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d48dc4-ec7c-40e3-a3a3-52aab879ae2a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.831737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 530/1057] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:49.833691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d9060-24c0-4d82-bd1a-102c9c64648a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:49.834101 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d9060-24c0-4d82-bd1a-102c9c64648a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:49.834526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d9060-24c0-4d82-bd1a-102c9c64648a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:49.859638 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ec2d9060-24c0-4d82-bd1a-102c9c64648a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:49.861882 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ec2d9060-24c0-4d82-bd1a-102c9c64648a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:49.907897 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 528/1058] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:45:49.914012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe700d-6d81-46ea-a38f-70f1765193dd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:49.945604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 531/1059] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:50.020647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:50.025771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fe700d-6d81-46ea-a38f-70f1765193dd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:50.061186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c3951-b992-4735-b563-4c779f510258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.061906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c3951-b992-4735-b563-4c779f510258 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.062231 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c3951-b992-4735-b563-4c779f510258 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.062509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c3951-b992-4735-b563-4c779f510258 None admin] PATH_INFO: `/v3/projects/ca3d870bb7204941a374655473836820` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.063617 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c3951-b992-4735-b563-4c779f510258 None admin] RBAC: Authorizing `identity:get_project(project_id=ca3d870bb7204941a374655473836820)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:50.065072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c3951-b992-4735-b563-4c779f510258 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:50.071516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 529/1060] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:49 2022] GET /identity/v3/projects/ca3d870bb7204941a374655473836820 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:50.072688 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d9060-24c0-4d82-bd1a-102c9c64648a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:50.089352 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:50.207606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d9060-24c0-4d82-bd1a-102c9c64648a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:50.228939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c3951-b992-4735-b563-4c779f510258 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:50.250427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c030e4-f218-4182-8b78-ce0ecefb92fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.251114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c030e4-f218-4182-8b78-ce0ecefb92fc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.251323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c030e4-f218-4182-8b78-ce0ecefb92fc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.251588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c030e4-f218-4182-8b78-ce0ecefb92fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.252347 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c030e4-f218-4182-8b78-ce0ecefb92fc None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=ca3d870bb7204941a374655473836820, user.id=59e68dadce234da3bcf75dcbce5582be) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:50.252649 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c030e4-f218-4182-8b78-ce0ecefb92fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:50.254062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c030e4-f218-4182-8b78-ce0ecefb92fc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:50.259724 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 532/1061] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:50 2022] GET /identity/v3/role_assignments?user.id=59e68dadce234da3bcf75dcbce5582be&scope.project.id=ca3d870bb7204941a374655473836820&role.id=07626899472b4e32adc0c71ce796a25d => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:50.355033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c3951-b992-4735-b563-4c779f510258 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:50.394831 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0190ff-6108-4422-8bfb-be355ad4da39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.395520 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0190ff-6108-4422-8bfb-be355ad4da39 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.395835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0190ff-6108-4422-8bfb-be355ad4da39 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.396111 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0190ff-6108-4422-8bfb-be355ad4da39 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.413449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0190ff-6108-4422-8bfb-be355ad4da39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-bc0190ff-6108-4422-8bfb-be355ad4da39 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:50.418469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:50.419425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:50.421107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:50.421107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:50.421107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:50.421107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:50.421107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:50.421107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:50.421107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:50.421107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:50.421107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:50.421107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:50.421107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:45:50.421107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:50.421107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 530/1062] 10.222.0.67 () {56 vars in 1120 bytes} [Mon Sep 26 17:45:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:50.428945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c030e4-f218-4182-8b78-ce0ecefb92fc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:50.511833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:50.534201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c030e4-f218-4182-8b78-ce0ecefb92fc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:50.569334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee57432-dd37-4126-95e6-a96067b6e47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.569946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee57432-dd37-4126-95e6-a96067b6e47a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.570225 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee57432-dd37-4126-95e6-a96067b6e47a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.570493 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee57432-dd37-4126-95e6-a96067b6e47a None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.571144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee57432-dd37-4126-95e6-a96067b6e47a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:50.571443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee57432-dd37-4126-95e6-a96067b6e47a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:50.572973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee57432-dd37-4126-95e6-a96067b6e47a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:50.586994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 533/1063] 10.222.0.67 () {56 vars in 1115 bytes} [Mon Sep 26 17:45:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:50.588450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0190ff-6108-4422-8bfb-be355ad4da39 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:50.619103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:50.694790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0190ff-6108-4422-8bfb-be355ad4da39 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:50.728947 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed18ecd-6cb9-4bb4-ab5d-4b2393ef7546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.729582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed18ecd-6cb9-4bb4-ab5d-4b2393ef7546 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.729857 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed18ecd-6cb9-4bb4-ab5d-4b2393ef7546 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.730126 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed18ecd-6cb9-4bb4-ab5d-4b2393ef7546 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.740457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed18ecd-6cb9-4bb4-ab5d-4b2393ef7546 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:50.741900 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed18ecd-6cb9-4bb4-ab5d-4b2393ef7546 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-aed18ecd-6cb9-4bb4-ab5d-4b2393ef7546 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:50.757713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:50.758643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:50.759515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:50.759515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:50.759515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:50.759515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 531/1064] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:50.765406 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee57432-dd37-4126-95e6-a96067b6e47a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:50.872886 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee57432-dd37-4126-95e6-a96067b6e47a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:50.912101 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7384c80-6550-48ee-94e0-89d243fc3e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:50.912720 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7384c80-6550-48ee-94e0-89d243fc3e8e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:50.912998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7384c80-6550-48ee-94e0-89d243fc3e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:50.913266 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7384c80-6550-48ee-94e0-89d243fc3e8e None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:50.913907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7384c80-6550-48ee-94e0-89d243fc3e8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:50.914206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7384c80-6550-48ee-94e0-89d243fc3e8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:50.915562 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7384c80-6550-48ee-94e0-89d243fc3e8e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:50.925309 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:50.929333 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 534/1065] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:50.934867 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed18ecd-6cb9-4bb4-ab5d-4b2393ef7546 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.002581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec66dba-a662-4dd8-a9cb-d78a49213e1f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.002901 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec66dba-a662-4dd8-a9cb-d78a49213e1f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.003147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec66dba-a662-4dd8-a9cb-d78a49213e1f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.005783 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 535/1066] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:51.009606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961475b-ad70-436f-8773-f714174314ef None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.009891 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961475b-ad70-436f-8773-f714174314ef None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.010166 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961475b-ad70-436f-8773-f714174314ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.026992 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:51.028457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8961475b-ad70-436f-8773-f714174314ef None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:51.029641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8961475b-ad70-436f-8773-f714174314ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:51.047759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed18ecd-6cb9-4bb4-ab5d-4b2393ef7546 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.084104 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2de48-db24-46df-b449-3b0f6d319c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.085074 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2de48-db24-46df-b449-3b0f6d319c6e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.085361 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2de48-db24-46df-b449-3b0f6d319c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.085639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f2de48-db24-46df-b449-3b0f6d319c6e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.093549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2de48-db24-46df-b449-3b0f6d319c6e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.095001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f2de48-db24-46df-b449-3b0f6d319c6e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-14f2de48-db24-46df-b449-3b0f6d319c6e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:51.107692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:51.109328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:51.110254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:51.110254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:51.110254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:51.110254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:51.110254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:51.110254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:51.110254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:51.110254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:51.110254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:45:51.110254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:51.110254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 532/1067] 10.222.0.67 () {56 vars in 1093 bytes} [Mon Sep 26 17:45:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:51.110850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 536/1068] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:51.113523 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2de48-db24-46df-b449-3b0f6d319c6e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.221945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f2de48-db24-46df-b449-3b0f6d319c6e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.257450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729c41b-6fe5-4093-811b-95c8bc093a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.258055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729c41b-6fe5-4093-811b-95c8bc093a2f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.258338 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729c41b-6fe5-4093-811b-95c8bc093a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.258615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c729c41b-6fe5-4093-811b-95c8bc093a2f None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.259339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729c41b-6fe5-4093-811b-95c8bc093a2f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:51.259660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729c41b-6fe5-4093-811b-95c8bc093a2f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.261100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c729c41b-6fe5-4093-811b-95c8bc093a2f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:51.273160 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:51.282185 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 533/1069] 10.222.0.67 () {56 vars in 1133 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:51.288152 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961475b-ad70-436f-8773-f714174314ef None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.439359 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961475b-ad70-436f-8773-f714174314ef None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.441929 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729c41b-6fe5-4093-811b-95c8bc093a2f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.473045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:51.479593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9b1b2-b953-44df-a6fb-de0cd1aede03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.480255 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9b1b2-b953-44df-a6fb-de0cd1aede03 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.480583 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9b1b2-b953-44df-a6fb-de0cd1aede03 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.487696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9b1b2-b953-44df-a6fb-de0cd1aede03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.491497 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9b1b2-b953-44df-a6fb-de0cd1aede03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.492970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9b1b2-b953-44df-a6fb-de0cd1aede03 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-18a9b1b2-b953-44df-a6fb-de0cd1aede03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:51.508679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:51.514633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:51.515739 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:51.515739 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:51.515739 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:51.515739 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 537/1070] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:51.515979 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9b1b2-b953-44df-a6fb-de0cd1aede03 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.572245 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c729c41b-6fe5-4093-811b-95c8bc093a2f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.610224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80d0b0-2428-4cef-8ac3-9563b60e991d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.610901 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80d0b0-2428-4cef-8ac3-9563b60e991d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.611109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80d0b0-2428-4cef-8ac3-9563b60e991d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.611385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee80d0b0-2428-4cef-8ac3-9563b60e991d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.633757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9b1b2-b953-44df-a6fb-de0cd1aede03 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.671745 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a7930-1e80-4745-b950-7b5ac49a6f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.672372 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a7930-1e80-4745-b950-7b5ac49a6f02 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.672656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a7930-1e80-4745-b950-7b5ac49a6f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.672922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a7930-1e80-4745-b950-7b5ac49a6f02 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.673561 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a7930-1e80-4745-b950-7b5ac49a6f02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:51.673864 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a7930-1e80-4745-b950-7b5ac49a6f02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.675241 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9a7930-1e80-4745-b950-7b5ac49a6f02 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:51.682045 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:51.685442 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80d0b0-2428-4cef-8ac3-9563b60e991d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.686912 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee80d0b0-2428-4cef-8ac3-9563b60e991d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:51.693830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 538/1071] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:51.701620 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a7930-1e80-4745-b950-7b5ac49a6f02 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.745019 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:51.793592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 534/1072] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 17:45:51.800340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80d0b0-2428-4cef-8ac3-9563b60e991d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.806306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a7930-1e80-4745-b950-7b5ac49a6f02 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.843619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49aa85a-d9a7-4fe6-9f3f-19a6e8322ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.844610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aa85a-d9a7-4fe6-9f3f-19a6e8322ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.844948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aa85a-d9a7-4fe6-9f3f-19a6e8322ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.845386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49aa85a-d9a7-4fe6-9f3f-19a6e8322ad5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.852278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49aa85a-d9a7-4fe6-9f3f-19a6e8322ad5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.853708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49aa85a-d9a7-4fe6-9f3f-19a6e8322ad5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:51.865969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b49aa85a-d9a7-4fe6-9f3f-19a6e8322ad5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:51.865969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:51.865969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:51.865969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:51.865969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:51.865969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:51.865969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:51.865969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:51.865969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:51.865969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:51.865969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:51.865969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:51.865969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:51.865969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:51.865969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:51.867130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:51.868247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:51.868247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:51.868247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:51.868247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:51.868247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:51.868247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:51.868247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:51.868247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:51.868247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 539/1073] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:51.872962 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49aa85a-d9a7-4fe6-9f3f-19a6e8322ad5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:51.908497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:51.913327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee80d0b0-2428-4cef-8ac3-9563b60e991d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.954570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205278c-ed03-4ea0-b844-93932a5e2689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:51.955206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205278c-ed03-4ea0-b844-93932a5e2689 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:51.955490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205278c-ed03-4ea0-b844-93932a5e2689 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:51.955795 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205278c-ed03-4ea0-b844-93932a5e2689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:51.967216 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a205278c-ed03-4ea0-b844-93932a5e2689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:51.969588 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a205278c-ed03-4ea0-b844-93932a5e2689 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a205278c-ed03-4ea0-b844-93932a5e2689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:51.985524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:51.986473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:51.987411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:51.987411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:51.987411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:51.987411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 535/1074] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:51.990528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49aa85a-d9a7-4fe6-9f3f-19a6e8322ad5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:51.992391 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:51.992572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205278c-ed03-4ea0-b844-93932a5e2689 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:52.027515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b59fc-f602-4481-9414-29525ba12a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:52.028077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b59fc-f602-4481-9414-29525ba12a2f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:52.028358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b59fc-f602-4481-9414-29525ba12a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:52.028631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b59fc-f602-4481-9414-29525ba12a2f None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:52.029314 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b59fc-f602-4481-9414-29525ba12a2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:52.029616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b59fc-f602-4481-9414-29525ba12a2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:52.032031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b59fc-f602-4481-9414-29525ba12a2f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:52.044507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 540/1075] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:52.051372 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b59fc-f602-4481-9414-29525ba12a2f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:52.106682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205278c-ed03-4ea0-b844-93932a5e2689 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:52.141481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b61e81f-152f-4575-b0b4-f44ffc40b324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:52.142081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e81f-152f-4575-b0b4-f44ffc40b324 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:52.142362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e81f-152f-4575-b0b4-f44ffc40b324 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:52.142630 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b61e81f-152f-4575-b0b4-f44ffc40b324 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:52.143264 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61e81f-152f-4575-b0b4-f44ffc40b324 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:52.143563 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61e81f-152f-4575-b0b4-f44ffc40b324 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:52.144958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b61e81f-152f-4575-b0b4-f44ffc40b324 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:52.164259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 536/1076] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:52.164506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b59fc-f602-4481-9414-29525ba12a2f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:52.201916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34823fb9-b1f7-4185-823d-671b9862c3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:52.202965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34823fb9-b1f7-4185-823d-671b9862c3a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:52.203250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34823fb9-b1f7-4185-823d-671b9862c3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:52.203525 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34823fb9-b1f7-4185-823d-671b9862c3a6 None admin] PATH_INFO: `/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/b9ed29a3f6be4d64bb3751f9e95eb56b/roles/fb6eaf230a474167b3a3726cbd2d35aa` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:52.205799 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34823fb9-b1f7-4185-823d-671b9862c3a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=71c96e5caeb44927bc21504fe54f0f98, user_id=b9ed29a3f6be4d64bb3751f9e95eb56b, role_id=fb6eaf230a474167b3a3726cbd2d35aa)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:52.207242 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34823fb9-b1f7-4185-823d-671b9862c3a6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:52.279917 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 541/1077] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:45:52 2022] PUT /identity/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/b9ed29a3f6be4d64bb3751f9e95eb56b/roles/fb6eaf230a474167b3a3726cbd2d35aa => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:52.465759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05849209-f889-44a1-afc8-877a3fa2017e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:52.466046 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05849209-f889-44a1-afc8-877a3fa2017e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:52.466318 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05849209-f889-44a1-afc8-877a3fa2017e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:52.468063 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 537/1078] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:52.473892 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325f6d-0e9c-4074-b507-d756afc14964 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:52.474759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325f6d-0e9c-4074-b507-d756afc14964 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:52.475254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36325f6d-0e9c-4074-b507-d756afc14964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:52.495784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-36325f6d-0e9c-4074-b507-d756afc14964 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:52.496871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-36325f6d-0e9c-4074-b507-d756afc14964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:52.514371 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:52.565483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 542/1079] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:52.835932 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05849209-f889-44a1-afc8-877a3fa2017e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:52.848825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:52.939949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05849209-f889-44a1-afc8-877a3fa2017e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:52.973246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff66a41-096b-4c4d-875c-6470a72cc7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:52.973868 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff66a41-096b-4c4d-875c-6470a72cc7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:52.974143 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff66a41-096b-4c4d-875c-6470a72cc7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:52.974410 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff66a41-096b-4c4d-875c-6470a72cc7b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:52.989479 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff66a41-096b-4c4d-875c-6470a72cc7b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9ff66a41-096b-4c4d-875c-6470a72cc7b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:52.994253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:52.995379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:52.996386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:52.996386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:52.996386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:52.996386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:52.996386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:52.996386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:52.996386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:52.996386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:52.996386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:52.996386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:52.996386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:52.996386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:52.996386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 538/1080] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:53.001908 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36325f6d-0e9c-4074-b507-d756afc14964 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:53.105291 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36325f6d-0e9c-4074-b507-d756afc14964 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:53.138622 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d7c3-86c1-4831-8d73-1a4c90561b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:53.139228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d7c3-86c1-4831-8d73-1a4c90561b34 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:53.140062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d7c3-86c1-4831-8d73-1a4c90561b34 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:53.140474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b823d7c3-86c1-4831-8d73-1a4c90561b34 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:53.141136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823d7c3-86c1-4831-8d73-1a4c90561b34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:53.141439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823d7c3-86c1-4831-8d73-1a4c90561b34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:53.142801 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b823d7c3-86c1-4831-8d73-1a4c90561b34 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:53.150665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 543/1081] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:53.157240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff66a41-096b-4c4d-875c-6470a72cc7b4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:53.230122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:53.260355 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff66a41-096b-4c4d-875c-6470a72cc7b4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:53.297009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01cb256-daae-4950-bad0-cc14120eadf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:53.297629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cb256-daae-4950-bad0-cc14120eadf1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:53.297948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cb256-daae-4950-bad0-cc14120eadf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:53.298221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01cb256-daae-4950-bad0-cc14120eadf1 None admin] PATH_INFO: `/v3/users/59e68dadce234da3bcf75dcbce5582be` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:53.299325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01cb256-daae-4950-bad0-cc14120eadf1 None admin] RBAC: Authorizing `identity:get_user(user_id=59e68dadce234da3bcf75dcbce5582be)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:53.300787 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01cb256-daae-4950-bad0-cc14120eadf1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:53.302346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 539/1082] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:53 2022] GET /identity/v3/users/59e68dadce234da3bcf75dcbce5582be => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:53.308099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d7c3-86c1-4831-8d73-1a4c90561b34 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:53.325568 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:53.409222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b823d7c3-86c1-4831-8d73-1a4c90561b34 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:53.442798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aec478a-68bc-4753-80c3-9c076e12c7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:53.443406 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec478a-68bc-4753-80c3-9c076e12c7d2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:53.443717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec478a-68bc-4753-80c3-9c076e12c7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:53.443994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aec478a-68bc-4753-80c3-9c076e12c7d2 None admin] PATH_INFO: `/v3/projects/ca3d870bb7204941a374655473836820` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:53.445102 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec478a-68bc-4753-80c3-9c076e12c7d2 None admin] RBAC: Authorizing `identity:get_project(project_id=ca3d870bb7204941a374655473836820)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:53.446516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aec478a-68bc-4753-80c3-9c076e12c7d2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:53.448187 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 544/1083] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:53 2022] GET /identity/v3/projects/ca3d870bb7204941a374655473836820 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:53.455219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01cb256-daae-4950-bad0-cc14120eadf1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:53.555539 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01cb256-daae-4950-bad0-cc14120eadf1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:53.590931 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7392dd-d753-41a1-b752-493a95ccc9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:53.591590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7392dd-d753-41a1-b752-493a95ccc9a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:53.591902 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7392dd-d753-41a1-b752-493a95ccc9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:53.592179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7392dd-d753-41a1-b752-493a95ccc9a3 None admin] PATH_INFO: `/v3/projects/ca3d870bb7204941a374655473836820/users/59e68dadce234da3bcf75dcbce5582be/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:53.594730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7392dd-d753-41a1-b752-493a95ccc9a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca3d870bb7204941a374655473836820, user_id=59e68dadce234da3bcf75dcbce5582be, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:53.596233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7392dd-d753-41a1-b752-493a95ccc9a3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:53.649625 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:53.654969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 540/1084] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:45:53 2022] PUT /identity/v3/projects/ca3d870bb7204941a374655473836820/users/59e68dadce234da3bcf75dcbce5582be/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:45:54.491042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728b04d-67b4-407a-8476-894dfa5ebede None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:54.491590 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728b04d-67b4-407a-8476-894dfa5ebede None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:54.491710 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f728b04d-67b4-407a-8476-894dfa5ebede None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:54.493058 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 545/1085] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:54.498309 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e99fa-11a9-4039-9f01-8076b5e47225 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:54.499277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e99fa-11a9-4039-9f01-8076b5e47225 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:54.499398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e99fa-11a9-4039-9f01-8076b5e47225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:54.518712 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-af5e99fa-11a9-4039-9f01-8076b5e47225 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:54.519970 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-af5e99fa-11a9-4039-9f01-8076b5e47225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:54.587857 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 541/1086] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:54.858884 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728b04d-67b4-407a-8476-894dfa5ebede None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:54.941829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:54.960374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f728b04d-67b4-407a-8476-894dfa5ebede None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:54.995242 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc4d04-e565-4c4e-b623-9d46d4f2d509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:54.995895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc4d04-e565-4c4e-b623-9d46d4f2d509 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:54.996174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc4d04-e565-4c4e-b623-9d46d4f2d509 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:54.996445 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cc4d04-e565-4c4e-b623-9d46d4f2d509 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.006832 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cc4d04-e565-4c4e-b623-9d46d4f2d509 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-98cc4d04-e565-4c4e-b623-9d46d4f2d509 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:55.011599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:55.012674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:55.013763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:55.013763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:55.013763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:55.013763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:55.013763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:55.013763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:55.013763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:55.013763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:55.013763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:55.013763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:55.013763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 26 17:45:55.013763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:55.013763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 546/1087] 10.222.0.67 () {56 vars in 1120 bytes} [Mon Sep 26 17:45:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:55.019528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e99fa-11a9-4039-9f01-8076b5e47225 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.116148 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e99fa-11a9-4039-9f01-8076b5e47225 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:55.141750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:55.149387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f3bc0-ad6d-450c-9c66-c75a90b3883c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.156331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f3bc0-ad6d-450c-9c66-c75a90b3883c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.156331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f3bc0-ad6d-450c-9c66-c75a90b3883c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.156331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563f3bc0-ad6d-450c-9c66-c75a90b3883c None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.156331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f3bc0-ad6d-450c-9c66-c75a90b3883c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:55.156331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f3bc0-ad6d-450c-9c66-c75a90b3883c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:55.156331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563f3bc0-ad6d-450c-9c66-c75a90b3883c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:55.161097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 542/1088] 10.222.0.67 () {56 vars in 1115 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:55.166968 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc4d04-e565-4c4e-b623-9d46d4f2d509 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.264703 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cc4d04-e565-4c4e-b623-9d46d4f2d509 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:55.299433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93e104-3619-4181-887a-8cbe68f54bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.300087 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93e104-3619-4181-887a-8cbe68f54bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.300365 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93e104-3619-4181-887a-8cbe68f54bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.300631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93e104-3619-4181-887a-8cbe68f54bd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.310570 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93e104-3619-4181-887a-8cbe68f54bd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:55.312049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd93e104-3619-4181-887a-8cbe68f54bd0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:55.313725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-fd93e104-3619-4181-887a-8cbe68f54bd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:55.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:55.328936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:55.329987 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:55.329987 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:55.329987 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:55.329987 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 547/1089] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:55.334813 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f3bc0-ad6d-450c-9c66-c75a90b3883c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.440000 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563f3bc0-ad6d-450c-9c66-c75a90b3883c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:55.478047 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0d59c-b04c-4b34-afe4-406b94235c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.478655 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0d59c-b04c-4b34-afe4-406b94235c53 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.478935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0d59c-b04c-4b34-afe4-406b94235c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.479207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0d59c-b04c-4b34-afe4-406b94235c53 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.479869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0d59c-b04c-4b34-afe4-406b94235c53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:55.480171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0d59c-b04c-4b34-afe4-406b94235c53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:55.481528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e0d59c-b04c-4b34-afe4-406b94235c53 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:55.493875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 543/1090] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:55.499506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93e104-3619-4181-887a-8cbe68f54bd0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.598253 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93e104-3619-4181-887a-8cbe68f54bd0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:55.630774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07665384-2b84-4d81-9184-5ff1730317a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.631398 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07665384-2b84-4d81-9184-5ff1730317a4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.631700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07665384-2b84-4d81-9184-5ff1730317a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.631969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07665384-2b84-4d81-9184-5ff1730317a4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.640033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07665384-2b84-4d81-9184-5ff1730317a4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:55.641478 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07665384-2b84-4d81-9184-5ff1730317a4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-07665384-2b84-4d81-9184-5ff1730317a4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:45:55.652334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:55.653441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:55.654741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:55.654741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:55.654741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:45:55.654741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:45:55.654741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:45:55.654741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:45:55.654741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:45:55.654741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:45:55.654741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 26 17:45:55.654741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:55.654741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 548/1091] 10.222.0.67 () {56 vars in 1093 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:55.659816 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0d59c-b04c-4b34-afe4-406b94235c53 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.710129 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:55.766195 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0d59c-b04c-4b34-afe4-406b94235c53 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:55.804534 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f78051c-0542-48a4-b706-37e51a33f0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.805144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78051c-0542-48a4-b706-37e51a33f0a2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.805425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78051c-0542-48a4-b706-37e51a33f0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.805699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f78051c-0542-48a4-b706-37e51a33f0a2 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.806426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78051c-0542-48a4-b706-37e51a33f0a2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:55.806734 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78051c-0542-48a4-b706-37e51a33f0a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:55.808202 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f78051c-0542-48a4-b706-37e51a33f0a2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:55.827330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 544/1092] 10.222.0.67 () {56 vars in 1133 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:55.833686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07665384-2b84-4d81-9184-5ff1730317a4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:55.864103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc96a60-2058-427d-8797-be4d7719e103 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.864383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc96a60-2058-427d-8797-be4d7719e103 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.864661 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc96a60-2058-427d-8797-be4d7719e103 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.865998 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 545/1093] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:45:55.871086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf215d9c-684e-4287-b496-bc3b56fc1bbf None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.871366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf215d9c-684e-4287-b496-bc3b56fc1bbf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.871639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf215d9c-684e-4287-b496-bc3b56fc1bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:55.891044 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cf215d9c-684e-4287-b496-bc3b56fc1bbf None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:55.892298 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cf215d9c-684e-4287-b496-bc3b56fc1bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:55.895478 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:55.940864 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07665384-2b84-4d81-9184-5ff1730317a4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:55.959107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 546/1094] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:45:55.974989 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57093b-e85d-4f88-a820-47b3f55498e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:55.978245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57093b-e85d-4f88-a820-47b3f55498e0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:55.978245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57093b-e85d-4f88-a820-47b3f55498e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:55.978245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f57093b-e85d-4f88-a820-47b3f55498e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.046168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57093b-e85d-4f88-a820-47b3f55498e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.047550 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f57093b-e85d-4f88-a820-47b3f55498e0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.105743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:56.160720 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 549/1095] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:45:56.166785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf215d9c-684e-4287-b496-bc3b56fc1bbf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.206735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:56.239597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57093b-e85d-4f88-a820-47b3f55498e0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.266979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf215d9c-684e-4287-b496-bc3b56fc1bbf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.303783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57128849-322f-48f5-86ad-b99a620df120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.304759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57128849-322f-48f5-86ad-b99a620df120 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.305038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57128849-322f-48f5-86ad-b99a620df120 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.305305 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57128849-322f-48f5-86ad-b99a620df120 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.315898 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57128849-322f-48f5-86ad-b99a620df120 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.317274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57128849-322f-48f5-86ad-b99a620df120 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-57128849-322f-48f5-86ad-b99a620df120 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:45:56.331783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:56.332725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:45:56.333605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:45:56.333605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:45:56.333605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:56.333605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 547/1096] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:45:56.338543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57128849-322f-48f5-86ad-b99a620df120 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.343028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f57093b-e85d-4f88-a820-47b3f55498e0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.377857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ffebf-6355-4e4d-a820-c44e430d8051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.378487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ffebf-6355-4e4d-a820-c44e430d8051 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.378768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ffebf-6355-4e4d-a820-c44e430d8051 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.379042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2ffebf-6355-4e4d-a820-c44e430d8051 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.389556 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2ffebf-6355-4e4d-a820-c44e430d8051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1c2ffebf-6355-4e4d-a820-c44e430d8051 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:45:56.394374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:56.395295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:56.396165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:56.396165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:56.396165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:56.396165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:56.396165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:45:56.396165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:45:56.396165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:45:56.396165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:45:56.396165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:45:56.396165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:45:56.396165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:45:56.396165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:45:56.396165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 550/1097] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:56.401442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ffebf-6355-4e4d-a820-c44e430d8051 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.416912 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:56.428661 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:56.438296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57128849-322f-48f5-86ad-b99a620df120 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.472455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604427e-1f0d-458e-891a-c58e1e7a6fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.473063 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604427e-1f0d-458e-891a-c58e1e7a6fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.473341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604427e-1f0d-458e-891a-c58e1e7a6fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.473615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604427e-1f0d-458e-891a-c58e1e7a6fa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.474258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604427e-1f0d-458e-891a-c58e1e7a6fa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:56.474560 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604427e-1f0d-458e-891a-c58e1e7a6fa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.475965 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604427e-1f0d-458e-891a-c58e1e7a6fa2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.488724 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 548/1098] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:56.494409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604427e-1f0d-458e-891a-c58e1e7a6fa2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.505562 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2ffebf-6355-4e4d-a820-c44e430d8051 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.540217 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bab0f-0b07-469d-bab1-459dcc43901a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.550267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bab0f-0b07-469d-bab1-459dcc43901a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.550267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bab0f-0b07-469d-bab1-459dcc43901a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.550267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6bab0f-0b07-469d-bab1-459dcc43901a None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.550267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bab0f-0b07-469d-bab1-459dcc43901a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:56.550267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bab0f-0b07-469d-bab1-459dcc43901a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.550267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6bab0f-0b07-469d-bab1-459dcc43901a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.552826 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 551/1099] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:56.558705 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bab0f-0b07-469d-bab1-459dcc43901a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.595592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604427e-1f0d-458e-891a-c58e1e7a6fa2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.631132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7b193-8be6-4361-aa68-c7ac62607cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.631785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7b193-8be6-4361-aa68-c7ac62607cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.632067 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7b193-8be6-4361-aa68-c7ac62607cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.632340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7b193-8be6-4361-aa68-c7ac62607cc1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.635458 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:56.639143 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7b193-8be6-4361-aa68-c7ac62607cc1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.641626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7b193-8be6-4361-aa68-c7ac62607cc1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.652656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e5a7b193-8be6-4361-aa68-c7ac62607cc1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:56.652656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:45:56.652656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:45:56.652656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:45:56.652656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:45:56.652656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:45:56.652656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:45:56.652656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:45:56.652656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:45:56.652656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:45:56.652656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:45:56.652656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:45:56.652656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:45:56.652656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:45:56.652656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:45:56.653674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:45:56.654557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:45:56.654557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:45:56.654557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:45:56.654557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:45:56.654557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:45:56.654557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:45:56.654557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:45:56.654557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:45:56.654557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 549/1100] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 26 17:45:56.659783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7b193-8be6-4361-aa68-c7ac62607cc1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.660241 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6bab0f-0b07-469d-bab1-459dcc43901a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.674824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:56.695262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0fcd7-a5ed-45fb-8fd7-a160539651a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.695941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0fcd7-a5ed-45fb-8fd7-a160539651a3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.696219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0fcd7-a5ed-45fb-8fd7-a160539651a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.696484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d0fcd7-a5ed-45fb-8fd7-a160539651a3 None admin] PATH_INFO: `/v3/users/59e68dadce234da3bcf75dcbce5582be` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.697634 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0fcd7-a5ed-45fb-8fd7-a160539651a3 None admin] RBAC: Authorizing `identity:get_user(user_id=59e68dadce234da3bcf75dcbce5582be)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.699150 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d0fcd7-a5ed-45fb-8fd7-a160539651a3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.700750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 552/1101] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/users/59e68dadce234da3bcf75dcbce5582be => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:56.707094 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0fcd7-a5ed-45fb-8fd7-a160539651a3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.762948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7b193-8be6-4361-aa68-c7ac62607cc1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.796888 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961412c-acac-48b1-a067-e86686c39b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.797493 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961412c-acac-48b1-a067-e86686c39b07 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.797774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961412c-acac-48b1-a067-e86686c39b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.798046 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9961412c-acac-48b1-a067-e86686c39b07 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.798726 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961412c-acac-48b1-a067-e86686c39b07 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:56.799036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961412c-acac-48b1-a067-e86686c39b07 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.800485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9961412c-acac-48b1-a067-e86686c39b07 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.804207 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d0fcd7-a5ed-45fb-8fd7-a160539651a3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.812750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 550/1102] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:56.818842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961412c-acac-48b1-a067-e86686c39b07 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:56.837931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd35566-8425-4327-a1f2-0e2aecc7a492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.838532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd35566-8425-4327-a1f2-0e2aecc7a492 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.838815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd35566-8425-4327-a1f2-0e2aecc7a492 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.839083 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd35566-8425-4327-a1f2-0e2aecc7a492 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.857418 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:56.887547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:56.912762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd35566-8425-4327-a1f2-0e2aecc7a492 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.914210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd35566-8425-4327-a1f2-0e2aecc7a492 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.919500 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9961412c-acac-48b1-a067-e86686c39b07 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:56.954317 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec167b8e-9bfa-4382-9d31-05378f4c933d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:56.954929 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec167b8e-9bfa-4382-9d31-05378f4c933d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:56.955192 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec167b8e-9bfa-4382-9d31-05378f4c933d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:56.955467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec167b8e-9bfa-4382-9d31-05378f4c933d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:56.956208 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec167b8e-9bfa-4382-9d31-05378f4c933d None admin] RBAC: Adding query filter params (role.id=fb6eaf230a474167b3a3726cbd2d35aa, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=b9ed29a3f6be4d64bb3751f9e95eb56b) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:56.956589 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec167b8e-9bfa-4382-9d31-05378f4c933d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:56.958028 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec167b8e-9bfa-4382-9d31-05378f4c933d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:56.963707 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 551/1103] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/role_assignments?user.id=b9ed29a3f6be4d64bb3751f9e95eb56b&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=fb6eaf230a474167b3a3726cbd2d35aa => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:45:57.016596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 553/1104] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:45:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 26 17:45:57.022614 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec167b8e-9bfa-4382-9d31-05378f4c933d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:57.130533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec167b8e-9bfa-4382-9d31-05378f4c933d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:57.173274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f26df-f9cc-424d-800d-f3c24ca4ca66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:57.173907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f26df-f9cc-424d-800d-f3c24ca4ca66 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:57.174186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f26df-f9cc-424d-800d-f3c24ca4ca66 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:57.174457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f26df-f9cc-424d-800d-f3c24ca4ca66 None admin] PATH_INFO: `/v3/projects/ca3d870bb7204941a374655473836820` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:57.175602 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f26df-f9cc-424d-800d-f3c24ca4ca66 None admin] RBAC: Authorizing `identity:get_project(project_id=ca3d870bb7204941a374655473836820)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:57.177063 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f26df-f9cc-424d-800d-f3c24ca4ca66 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:57.178683 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 552/1105] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:45:57 2022] GET /identity/v3/projects/ca3d870bb7204941a374655473836820 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:57.184962 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd35566-8425-4327-a1f2-0e2aecc7a492 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:57.234211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:45:57.289532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd35566-8425-4327-a1f2-0e2aecc7a492 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:45:57.334867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83148d5f-f5bb-4a8e-8d1a-db88f2ed8698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:45:57.335480 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83148d5f-f5bb-4a8e-8d1a-db88f2ed8698 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:57.335792 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83148d5f-f5bb-4a8e-8d1a-db88f2ed8698 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:57.336065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83148d5f-f5bb-4a8e-8d1a-db88f2ed8698 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:57.336784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83148d5f-f5bb-4a8e-8d1a-db88f2ed8698 None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=ca3d870bb7204941a374655473836820, user.id=59e68dadce234da3bcf75dcbce5582be) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:45:57.337088 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83148d5f-f5bb-4a8e-8d1a-db88f2ed8698 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:45:57.338514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83148d5f-f5bb-4a8e-8d1a-db88f2ed8698 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:45:57.345653 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 554/1106] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:45:57 2022] GET /identity/v3/role_assignments?user.id=59e68dadce234da3bcf75dcbce5582be&scope.project.id=ca3d870bb7204941a374655473836820&role.id=07626899472b4e32adc0c71ce796a25d => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:45:59.622745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5cf51-74ec-47dd-a2da-c70747b99f16 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:59.623295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5cf51-74ec-47dd-a2da-c70747b99f16 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:59.623295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5cf51-74ec-47dd-a2da-c70747b99f16 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:59.625055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 553/1107] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:45:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:45:59.630229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4744d-dfa7-48db-8ad7-d4928c9871d4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:45:59.630624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4744d-dfa7-48db-8ad7-d4928c9871d4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:45:59.630919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4744d-dfa7-48db-8ad7-d4928c9871d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:45:59.649236 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bfb4744d-dfa7-48db-8ad7-d4928c9871d4 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:45:59.650411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bfb4744d-dfa7-48db-8ad7-d4928c9871d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:45:59.717020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 555/1108] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:45:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:45:59.992508 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5cf51-74ec-47dd-a2da-c70747b99f16 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:45:59.995418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:00.096150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5cf51-74ec-47dd-a2da-c70747b99f16 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:00.129983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a027b2a-f01f-4ce6-a486-9cf64e2d5f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:00.130594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a027b2a-f01f-4ce6-a486-9cf64e2d5f74 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:00.130872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a027b2a-f01f-4ce6-a486-9cf64e2d5f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:00.131143 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a027b2a-f01f-4ce6-a486-9cf64e2d5f74 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:00.132242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a027b2a-f01f-4ce6-a486-9cf64e2d5f74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:00.133687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a027b2a-f01f-4ce6-a486-9cf64e2d5f74 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:00.135220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 554/1109] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:45:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:00.141207 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4744d-dfa7-48db-8ad7-d4928c9871d4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:00.179193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:00.238169 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4744d-dfa7-48db-8ad7-d4928c9871d4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:00.270422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700223d2-114c-4d9b-ac01-c97ddea38819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:00.271010 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700223d2-114c-4d9b-ac01-c97ddea38819 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:00.271285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700223d2-114c-4d9b-ac01-c97ddea38819 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:00.271551 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700223d2-114c-4d9b-ac01-c97ddea38819 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:00.272225 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700223d2-114c-4d9b-ac01-c97ddea38819 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:00.273627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700223d2-114c-4d9b-ac01-c97ddea38819 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:00.346407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 556/1110] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:46:00 2022] POST /identity/v3/users => generated 286 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:02.510996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c007bbd-8a08-4b80-ba64-44e33660a0df None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:02.511578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c007bbd-8a08-4b80-ba64-44e33660a0df None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:02.511578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c007bbd-8a08-4b80-ba64-44e33660a0df None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:02.513112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 555/1111] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:46:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:02.519058 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34561d-d7c1-4a94-b695-9d337cd36692 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:02.519473 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34561d-d7c1-4a94-b695-9d337cd36692 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:02.519586 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab34561d-d7c1-4a94-b695-9d337cd36692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:02.537553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ab34561d-d7c1-4a94-b695-9d337cd36692 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:02.538726 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ab34561d-d7c1-4a94-b695-9d337cd36692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:02.603056 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 557/1112] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:02.871189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c007bbd-8a08-4b80-ba64-44e33660a0df None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:02.939948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:02.971118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c007bbd-8a08-4b80-ba64-44e33660a0df None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:03.007456 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e713a-8b32-4a77-8723-c01490803b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:03.008113 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e713a-8b32-4a77-8723-c01490803b79 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.008392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e713a-8b32-4a77-8723-c01490803b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.008663 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e713a-8b32-4a77-8723-c01490803b79 None admin] PATH_INFO: `/v3/roles/30bb990dfe5c45fbbfed21c08c7859b4` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.010034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e713a-8b32-4a77-8723-c01490803b79 None admin] RBAC: Authorizing `identity:get_role(role_id=30bb990dfe5c45fbbfed21c08c7859b4)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:03.011394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e713a-8b32-4a77-8723-c01490803b79 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:03.012758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 556/1113] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:46:02 2022] GET /identity/v3/roles/30bb990dfe5c45fbbfed21c08c7859b4 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:03.018819 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34561d-d7c1-4a94-b695-9d337cd36692 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:03.042505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:03.116213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab34561d-d7c1-4a94-b695-9d337cd36692 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:03.149030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e500fd-7668-438c-9dc3-59bd32e940ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:03.149599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e500fd-7668-438c-9dc3-59bd32e940ed None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.149862 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e500fd-7668-438c-9dc3-59bd32e940ed None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.150129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e500fd-7668-438c-9dc3-59bd32e940ed None admin] PATH_INFO: `/v3/users/c212f754f1da43d989efc7847879029e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.162539 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e500fd-7668-438c-9dc3-59bd32e940ed None admin] RBAC: Authorizing `identity:get_user(user_id=c212f754f1da43d989efc7847879029e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:03.163992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e500fd-7668-438c-9dc3-59bd32e940ed None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:03.165555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 558/1114] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:46:03 2022] GET /identity/v3/users/c212f754f1da43d989efc7847879029e => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:03.171951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e713a-8b32-4a77-8723-c01490803b79 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:03.269145 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e713a-8b32-4a77-8723-c01490803b79 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:03.302282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247974a-36d3-434d-b6e5-5120fc3a3456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:03.302870 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247974a-36d3-434d-b6e5-5120fc3a3456 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.303146 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247974a-36d3-434d-b6e5-5120fc3a3456 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.303413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247974a-36d3-434d-b6e5-5120fc3a3456 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.311554 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:03.369824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247974a-36d3-434d-b6e5-5120fc3a3456 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:03.371268 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6247974a-36d3-434d-b6e5-5120fc3a3456 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:03.466884 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 557/1115] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:46:03.473474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e500fd-7668-438c-9dc3-59bd32e940ed None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:03.553304 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:03.570521 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e500fd-7668-438c-9dc3-59bd32e940ed None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:03.604213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef549b05-17fa-47bd-8cf1-2c0206ff6e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:03.604837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef549b05-17fa-47bd-8cf1-2c0206ff6e42 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.605118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef549b05-17fa-47bd-8cf1-2c0206ff6e42 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.605391 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef549b05-17fa-47bd-8cf1-2c0206ff6e42 None admin] PATH_INFO: `/v3/projects/ca3d870bb7204941a374655473836820` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.606491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef549b05-17fa-47bd-8cf1-2c0206ff6e42 None admin] RBAC: Authorizing `identity:get_project(project_id=ca3d870bb7204941a374655473836820)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:03.608504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef549b05-17fa-47bd-8cf1-2c0206ff6e42 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:03.610101 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 559/1116] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:46:03 2022] GET /identity/v3/projects/ca3d870bb7204941a374655473836820 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:03.616373 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247974a-36d3-434d-b6e5-5120fc3a3456 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:03.655299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:03.715143 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6247974a-36d3-434d-b6e5-5120fc3a3456 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:03.749342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea7b226-e985-42c1-b967-f90e2eb1e06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:03.749876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7b226-e985-42c1-b967-f90e2eb1e06b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:03.750158 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7b226-e985-42c1-b967-f90e2eb1e06b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:03.750448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea7b226-e985-42c1-b967-f90e2eb1e06b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:03.751150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7b226-e985-42c1-b967-f90e2eb1e06b None admin] RBAC: Adding query filter params (role.id=30bb990dfe5c45fbbfed21c08c7859b4, scope.project.id=ca3d870bb7204941a374655473836820, user.id=c212f754f1da43d989efc7847879029e) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:03.751462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7b226-e985-42c1-b967-f90e2eb1e06b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:03.752914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea7b226-e985-42c1-b967-f90e2eb1e06b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:03.758168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 558/1117] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:03 2022] GET /identity/v3/role_assignments?user.id=c212f754f1da43d989efc7847879029e&scope.project.id=ca3d870bb7204941a374655473836820&role.id=30bb990dfe5c45fbbfed21c08c7859b4 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:05.879482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161102d0-d30e-497b-89c8-03c45247e21f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:05.880201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161102d0-d30e-497b-89c8-03c45247e21f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:05.880201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161102d0-d30e-497b-89c8-03c45247e21f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:05.881400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 560/1118] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:46:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:05.886570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc69c00-e7a7-4bad-b6a9-a4df16d825cb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:05.887004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc69c00-e7a7-4bad-b6a9-a4df16d825cb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:05.887104 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc69c00-e7a7-4bad-b6a9-a4df16d825cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:05.906368 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cbc69c00-e7a7-4bad-b6a9-a4df16d825cb None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:05.907532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cbc69c00-e7a7-4bad-b6a9-a4df16d825cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:05.971666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 559/1119] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:06.239452 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161102d0-d30e-497b-89c8-03c45247e21f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:06.337971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161102d0-d30e-497b-89c8-03c45247e21f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:06.371305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670784f-3dc3-49c3-95ea-bf576e79fcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:06.371947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670784f-3dc3-49c3-95ea-bf576e79fcbd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:06.372228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670784f-3dc3-49c3-95ea-bf576e79fcbd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:06.372502 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670784f-3dc3-49c3-95ea-bf576e79fcbd None admin] PATH_INFO: `/v3/roles/30bb990dfe5c45fbbfed21c08c7859b4` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:06.374413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670784f-3dc3-49c3-95ea-bf576e79fcbd None admin] RBAC: Authorizing `identity:get_role(role_id=30bb990dfe5c45fbbfed21c08c7859b4)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:06.375812 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670784f-3dc3-49c3-95ea-bf576e79fcbd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:06.377126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 561/1120] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:46:06 2022] GET /identity/v3/roles/30bb990dfe5c45fbbfed21c08c7859b4 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:06.383084 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc69c00-e7a7-4bad-b6a9-a4df16d825cb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:06.414754 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:06.482357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc69c00-e7a7-4bad-b6a9-a4df16d825cb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:06.516795 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38482e72-6d41-4677-88e8-92487171ebc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:06.517354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38482e72-6d41-4677-88e8-92487171ebc0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:06.517636 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38482e72-6d41-4677-88e8-92487171ebc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:06.517908 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38482e72-6d41-4677-88e8-92487171ebc0 None admin] PATH_INFO: `/v3/users/c212f754f1da43d989efc7847879029e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:06.519012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38482e72-6d41-4677-88e8-92487171ebc0 None admin] RBAC: Authorizing `identity:get_user(user_id=c212f754f1da43d989efc7847879029e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:06.520467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38482e72-6d41-4677-88e8-92487171ebc0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:06.522019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 560/1121] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:46:06 2022] GET /identity/v3/users/c212f754f1da43d989efc7847879029e => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:06.527777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670784f-3dc3-49c3-95ea-bf576e79fcbd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:06.546315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:06.625643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670784f-3dc3-49c3-95ea-bf576e79fcbd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:06.658065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2b42d9-8ea1-4863-9622-dc8a2daa9862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:06.658679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b42d9-8ea1-4863-9622-dc8a2daa9862 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:06.658954 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b42d9-8ea1-4863-9622-dc8a2daa9862 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:06.659223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2b42d9-8ea1-4863-9622-dc8a2daa9862 None admin] PATH_INFO: `/v3/projects/ca3d870bb7204941a374655473836820` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:06.660328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b42d9-8ea1-4863-9622-dc8a2daa9862 None admin] RBAC: Authorizing `identity:get_project(project_id=ca3d870bb7204941a374655473836820)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:06.661735 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2b42d9-8ea1-4863-9622-dc8a2daa9862 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:06.663330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 562/1122] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:46:06 2022] GET /identity/v3/projects/ca3d870bb7204941a374655473836820 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:06.669239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38482e72-6d41-4677-88e8-92487171ebc0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:06.766265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:06.769175 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38482e72-6d41-4677-88e8-92487171ebc0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:06.809612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a2dc0-4941-46d6-a67a-649d4b756665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:06.810262 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a2dc0-4941-46d6-a67a-649d4b756665 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:06.810734 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a2dc0-4941-46d6-a67a-649d4b756665 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:06.811020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a2dc0-4941-46d6-a67a-649d4b756665 None admin] PATH_INFO: `/v3/projects/ca3d870bb7204941a374655473836820/users/c212f754f1da43d989efc7847879029e/roles/30bb990dfe5c45fbbfed21c08c7859b4` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:06.813159 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a2dc0-4941-46d6-a67a-649d4b756665 None admin] RBAC: Authorizing `identity:create_grant(project_id=ca3d870bb7204941a374655473836820, user_id=c212f754f1da43d989efc7847879029e, role_id=30bb990dfe5c45fbbfed21c08c7859b4)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:06.814592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a2dc0-4941-46d6-a67a-649d4b756665 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:06.859442 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 561/1123] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:46:06 2022] PUT /identity/v3/projects/ca3d870bb7204941a374655473836820/users/c212f754f1da43d989efc7847879029e/roles/30bb990dfe5c45fbbfed21c08c7859b4 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:46:09.012408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc9335-b19b-4779-95c4-3e4660a15d72 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:09.012837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc9335-b19b-4779-95c4-3e4660a15d72 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:09.012926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc9335-b19b-4779-95c4-3e4660a15d72 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:09.014630 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 563/1124] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:46:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:09.019880 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337f708-ec71-4868-90bc-8233a8370b09 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:09.020288 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337f708-ec71-4868-90bc-8233a8370b09 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:09.020427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e337f708-ec71-4868-90bc-8233a8370b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:09.038893 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e337f708-ec71-4868-90bc-8233a8370b09 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:09.040096 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e337f708-ec71-4868-90bc-8233a8370b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:09.104882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 562/1125] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:09.375265 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc9335-b19b-4779-95c4-3e4660a15d72 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:09.460637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:09.475563 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc9335-b19b-4779-95c4-3e4660a15d72 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:09.508438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023c9c5-4527-41ec-a7ae-7c5574a217e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:09.509055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023c9c5-4527-41ec-a7ae-7c5574a217e0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:09.509335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023c9c5-4527-41ec-a7ae-7c5574a217e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:09.509606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3023c9c5-4527-41ec-a7ae-7c5574a217e0 None admin] PATH_INFO: `/v3/roles/30bb990dfe5c45fbbfed21c08c7859b4` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:09.510973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023c9c5-4527-41ec-a7ae-7c5574a217e0 None admin] RBAC: Authorizing `identity:get_role(role_id=30bb990dfe5c45fbbfed21c08c7859b4)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:09.512360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3023c9c5-4527-41ec-a7ae-7c5574a217e0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:09.514520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 564/1126] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:46:09 2022] GET /identity/v3/roles/30bb990dfe5c45fbbfed21c08c7859b4 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:09.520291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337f708-ec71-4868-90bc-8233a8370b09 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:09.606559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:09.616118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e337f708-ec71-4868-90bc-8233a8370b09 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:09.650145 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418301e7-e72f-4cbd-b38f-80d07f51bd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:09.650707 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418301e7-e72f-4cbd-b38f-80d07f51bd9c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:09.650987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418301e7-e72f-4cbd-b38f-80d07f51bd9c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:09.651261 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418301e7-e72f-4cbd-b38f-80d07f51bd9c None admin] PATH_INFO: `/v3/users/c212f754f1da43d989efc7847879029e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:09.652400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418301e7-e72f-4cbd-b38f-80d07f51bd9c None admin] RBAC: Authorizing `identity:get_user(user_id=c212f754f1da43d989efc7847879029e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:09.653839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418301e7-e72f-4cbd-b38f-80d07f51bd9c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:09.655533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 563/1127] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:46:09 2022] GET /identity/v3/users/c212f754f1da43d989efc7847879029e => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:09.661439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023c9c5-4527-41ec-a7ae-7c5574a217e0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:09.758903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3023c9c5-4527-41ec-a7ae-7c5574a217e0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:09.792464 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87fc86-d2e0-48f8-abf0-6bc067fcdc52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:09.793060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87fc86-d2e0-48f8-abf0-6bc067fcdc52 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:09.793342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87fc86-d2e0-48f8-abf0-6bc067fcdc52 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:09.793615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87fc86-d2e0-48f8-abf0-6bc067fcdc52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:09.816136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:09.858750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b87fc86-d2e0-48f8-abf0-6bc067fcdc52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:09.860226 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b87fc86-d2e0-48f8-abf0-6bc067fcdc52 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:09.955098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 565/1128] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:46:09.960860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418301e7-e72f-4cbd-b38f-80d07f51bd9c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:10.060040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418301e7-e72f-4cbd-b38f-80d07f51bd9c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:10.093784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842f2bb9-3457-493a-a3d1-866e88340a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:10.094410 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f2bb9-3457-493a-a3d1-866e88340a79 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:10.094694 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f2bb9-3457-493a-a3d1-866e88340a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:10.094972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842f2bb9-3457-493a-a3d1-866e88340a79 None admin] PATH_INFO: `/v3/projects/ca3d870bb7204941a374655473836820` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:10.096106 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f2bb9-3457-493a-a3d1-866e88340a79 None admin] RBAC: Authorizing `identity:get_project(project_id=ca3d870bb7204941a374655473836820)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:10.097537 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842f2bb9-3457-493a-a3d1-866e88340a79 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:10.099157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 564/1129] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:46:09 2022] GET /identity/v3/projects/ca3d870bb7204941a374655473836820 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:10.105249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87fc86-d2e0-48f8-abf0-6bc067fcdc52 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:10.159054 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:10.202400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87fc86-d2e0-48f8-abf0-6bc067fcdc52 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:10.236045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da743031-8395-4abb-a4b6-c66358655797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:10.236581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da743031-8395-4abb-a4b6-c66358655797 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:10.236861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da743031-8395-4abb-a4b6-c66358655797 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:10.237131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da743031-8395-4abb-a4b6-c66358655797 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:10.237837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da743031-8395-4abb-a4b6-c66358655797 None admin] RBAC: Adding query filter params (role.id=30bb990dfe5c45fbbfed21c08c7859b4, scope.project.id=ca3d870bb7204941a374655473836820, user.id=c212f754f1da43d989efc7847879029e) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:10.238143 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da743031-8395-4abb-a4b6-c66358655797 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:10.239556 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da743031-8395-4abb-a4b6-c66358655797 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:10.245112 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 566/1130] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:10 2022] GET /identity/v3/role_assignments?user.id=c212f754f1da43d989efc7847879029e&scope.project.id=ca3d870bb7204941a374655473836820&role.id=30bb990dfe5c45fbbfed21c08c7859b4 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:12.398785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735684a-1ff3-4b70-b315-9d13acacf676 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:12.399337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735684a-1ff3-4b70-b315-9d13acacf676 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:12.399337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d735684a-1ff3-4b70-b315-9d13acacf676 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:12.400673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 565/1131] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:46:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:12.406328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b7944-5b31-4c70-9729-93a69909cb65 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:12.406747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b7944-5b31-4c70-9729-93a69909cb65 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:12.406886 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b7944-5b31-4c70-9729-93a69909cb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:12.425129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a71b7944-5b31-4c70-9729-93a69909cb65 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:12.426310 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a71b7944-5b31-4c70-9729-93a69909cb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:12.492431 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 567/1132] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:12.766875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735684a-1ff3-4b70-b315-9d13acacf676 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:12.793748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:12.865281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d735684a-1ff3-4b70-b315-9d13acacf676 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:12.898396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748949ba-8f75-41f7-aca4-a3514942c263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:12.899009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748949ba-8f75-41f7-aca4-a3514942c263 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:12.899283 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748949ba-8f75-41f7-aca4-a3514942c263 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:12.899551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748949ba-8f75-41f7-aca4-a3514942c263 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:12.900664 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748949ba-8f75-41f7-aca4-a3514942c263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:12.902076 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748949ba-8f75-41f7-aca4-a3514942c263 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:12.903640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 566/1133] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:12.910194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b7944-5b31-4c70-9729-93a69909cb65 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:12.957573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:13.007974 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b7944-5b31-4c70-9729-93a69909cb65 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:13.041218 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b3198-6aa8-4f08-9674-35757c5833ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:13.041754 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b3198-6aa8-4f08-9674-35757c5833ec None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:13.042032 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b3198-6aa8-4f08-9674-35757c5833ec None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:13.042303 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b3198-6aa8-4f08-9674-35757c5833ec None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:13.042939 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b3198-6aa8-4f08-9674-35757c5833ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:13.044376 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b3198-6aa8-4f08-9674-35757c5833ec None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:13.141240 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 568/1134] 10.222.0.67 () {60 vars in 1141 bytes} [Mon Sep 26 17:46:12 2022] POST /identity/v3/projects => generated 311 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:15.312779 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708234b2-be22-4c8b-8b23-93ecd7984b71 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:15.313294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708234b2-be22-4c8b-8b23-93ecd7984b71 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:15.313294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708234b2-be22-4c8b-8b23-93ecd7984b71 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:15.314561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 567/1135] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:46:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:15.319839 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d944a0-62e4-41d0-a826-d4117d658d7c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:15.320238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d944a0-62e4-41d0-a826-d4117d658d7c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:15.320392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d944a0-62e4-41d0-a826-d4117d658d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:15.338222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b7d944a0-62e4-41d0-a826-d4117d658d7c None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:15.339415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b7d944a0-62e4-41d0-a826-d4117d658d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:15.405275 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 569/1136] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:15.679038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708234b2-be22-4c8b-8b23-93ecd7984b71 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:15.771007 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:15.778367 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708234b2-be22-4c8b-8b23-93ecd7984b71 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:15.812094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea334c65-6f8f-4aba-b6db-a62cdcc7fc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:15.812714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea334c65-6f8f-4aba-b6db-a62cdcc7fc3d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:15.812989 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea334c65-6f8f-4aba-b6db-a62cdcc7fc3d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:15.813258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea334c65-6f8f-4aba-b6db-a62cdcc7fc3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:15.814343 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea334c65-6f8f-4aba-b6db-a62cdcc7fc3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:15.815849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea334c65-6f8f-4aba-b6db-a62cdcc7fc3d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:15.817384 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 568/1137] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:46:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:15.823118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d944a0-62e4-41d0-a826-d4117d658d7c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:15.853024 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:15.921226 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d944a0-62e4-41d0-a826-d4117d658d7c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:15.954254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a78325e-3661-4fb9-a13c-a1f8b6ab196a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:15.954797 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78325e-3661-4fb9-a13c-a1f8b6ab196a None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:15.955080 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78325e-3661-4fb9-a13c-a1f8b6ab196a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:15.955355 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a78325e-3661-4fb9-a13c-a1f8b6ab196a None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:15.956027 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78325e-3661-4fb9-a13c-a1f8b6ab196a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:15.957447 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a78325e-3661-4fb9-a13c-a1f8b6ab196a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:16.036771 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 570/1138] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:46:15 2022] POST /identity/v3/users => generated 286 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:18.229389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb6c7d-0d8e-47a8-9dc8-ce80c2fc79b9 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.229959 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb6c7d-0d8e-47a8-9dc8-ce80c2fc79b9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.229959 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0eb6c7d-0d8e-47a8-9dc8-ce80c2fc79b9 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.231270 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 569/1139] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:46:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:18.237277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb09f50-cfaf-432d-89ce-b43ce76cff1a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.237707 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb09f50-cfaf-432d-89ce-b43ce76cff1a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.237829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb09f50-cfaf-432d-89ce-b43ce76cff1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.255765 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-dfb09f50-cfaf-432d-89ce-b43ce76cff1a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:18.256956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dfb09f50-cfaf-432d-89ce-b43ce76cff1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:18.321558 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 571/1140] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:18.594764 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb6c7d-0d8e-47a8-9dc8-ce80c2fc79b9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:18.694770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0eb6c7d-0d8e-47a8-9dc8-ce80c2fc79b9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:18.728522 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10e12b-b25d-4463-82e4-4888a7cd35db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:18.729140 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10e12b-b25d-4463-82e4-4888a7cd35db None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.729420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10e12b-b25d-4463-82e4-4888a7cd35db None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.729695 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e10e12b-b25d-4463-82e4-4888a7cd35db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.739664 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e10e12b-b25d-4463-82e4-4888a7cd35db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0e10e12b-b25d-4463-82e4-4888a7cd35db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:18.744531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:18.745604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:18.746615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:18.746615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:18.746615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:18.746615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:18.746615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:18.746615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:18.746615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:18.746615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:18.746615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:18.746615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:18.746615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:18.746615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:46:18.746615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 570/1141] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:18.751876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb09f50-cfaf-432d-89ce-b43ce76cff1a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:18.771944 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:18.852301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb09f50-cfaf-432d-89ce-b43ce76cff1a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:18.884216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1aa2e-9108-4dad-a279-1a71e6d1aa29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:18.884810 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1aa2e-9108-4dad-a279-1a71e6d1aa29 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:18.885088 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1aa2e-9108-4dad-a279-1a71e6d1aa29 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:18.885354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb1aa2e-9108-4dad-a279-1a71e6d1aa29 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:18.885996 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1aa2e-9108-4dad-a279-1a71e6d1aa29 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:18.886293 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1aa2e-9108-4dad-a279-1a71e6d1aa29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:18.887640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb1aa2e-9108-4dad-a279-1a71e6d1aa29 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:18.896409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 572/1142] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:18.902315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10e12b-b25d-4463-82e4-4888a7cd35db None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:18.928126 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:19.005746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e10e12b-b25d-4463-82e4-4888a7cd35db None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:19.038943 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1561a1e-d0a2-4243-b332-5c7cbe66d6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:19.039559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1561a1e-d0a2-4243-b332-5c7cbe66d6ea None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:19.039863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1561a1e-d0a2-4243-b332-5c7cbe66d6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:19.040136 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1561a1e-d0a2-4243-b332-5c7cbe66d6ea None admin] PATH_INFO: `/v3/users/ccee017e80664ccda2b54e6de3f223e5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:19.052556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1561a1e-d0a2-4243-b332-5c7cbe66d6ea None admin] RBAC: Authorizing `identity:get_user(user_id=ccee017e80664ccda2b54e6de3f223e5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:19.053981 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1561a1e-d0a2-4243-b332-5c7cbe66d6ea None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:19.055559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 571/1143] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:46:18 2022] GET /identity/v3/users/ccee017e80664ccda2b54e6de3f223e5 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:19.061975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1aa2e-9108-4dad-a279-1a71e6d1aa29 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:19.139073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:19.159779 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb1aa2e-9108-4dad-a279-1a71e6d1aa29 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:19.192685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b7600-fcca-43de-8bb3-7fbc1671bf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:19.193330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b7600-fcca-43de-8bb3-7fbc1671bf31 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:19.193611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b7600-fcca-43de-8bb3-7fbc1671bf31 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:19.193879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b7600-fcca-43de-8bb3-7fbc1671bf31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:19.275751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b7600-fcca-43de-8bb3-7fbc1671bf31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:19.277259 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2b7600-fcca-43de-8bb3-7fbc1671bf31 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:19.355892 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:19.377861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 573/1144] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:46:19.383889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1561a1e-d0a2-4243-b332-5c7cbe66d6ea None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:19.470108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:19.489012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1561a1e-d0a2-4243-b332-5c7cbe66d6ea None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:19.522753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8771e18-980f-44a6-8dd8-1ecc7850c26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:19.523366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8771e18-980f-44a6-8dd8-1ecc7850c26b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:19.523640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8771e18-980f-44a6-8dd8-1ecc7850c26b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:19.523969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8771e18-980f-44a6-8dd8-1ecc7850c26b None admin] PATH_INFO: `/v3/projects/b65699c9f08a4497a810e9acb5966b92` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:19.525049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8771e18-980f-44a6-8dd8-1ecc7850c26b None admin] RBAC: Authorizing `identity:get_project(project_id=b65699c9f08a4497a810e9acb5966b92)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:19.526462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8771e18-980f-44a6-8dd8-1ecc7850c26b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:19.528156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 572/1145] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:46:19 2022] GET /identity/v3/projects/b65699c9f08a4497a810e9acb5966b92 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:19.534766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b7600-fcca-43de-8bb3-7fbc1671bf31 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:19.640201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b7600-fcca-43de-8bb3-7fbc1671bf31 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:19.673637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1608a5d-2729-4592-a972-1fe932068aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:19.674247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1608a5d-2729-4592-a972-1fe932068aca None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:19.674526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1608a5d-2729-4592-a972-1fe932068aca None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:19.674799 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1608a5d-2729-4592-a972-1fe932068aca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:19.675506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1608a5d-2729-4592-a972-1fe932068aca None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=b65699c9f08a4497a810e9acb5966b92, user.id=ccee017e80664ccda2b54e6de3f223e5) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:19.675837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1608a5d-2729-4592-a972-1fe932068aca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:19.677258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1608a5d-2729-4592-a972-1fe932068aca None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:19.682450 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 574/1146] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:19 2022] GET /identity/v3/role_assignments?user.id=ccee017e80664ccda2b54e6de3f223e5&scope.project.id=b65699c9f08a4497a810e9acb5966b92&role.id=07626899472b4e32adc0c71ce796a25d => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:21.783974 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e98f4-582c-4989-b47d-ea605e7b82bc None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:21.784537 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e98f4-582c-4989-b47d-ea605e7b82bc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:21.784537 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e98f4-582c-4989-b47d-ea605e7b82bc None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:21.785872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 573/1147] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:46:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:21.790980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d605a3-0515-47ab-8476-b8f2bb160c1d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:21.791377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d605a3-0515-47ab-8476-b8f2bb160c1d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:21.791533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d605a3-0515-47ab-8476-b8f2bb160c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:21.809573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-81d605a3-0515-47ab-8476-b8f2bb160c1d None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:21.810760 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-81d605a3-0515-47ab-8476-b8f2bb160c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:21.845803 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:21.876957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 575/1148] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:22.145188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e98f4-582c-4989-b47d-ea605e7b82bc None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:22.246238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e98f4-582c-4989-b47d-ea605e7b82bc None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:22.283327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc93ff-9f81-4c33-a6c6-e1a995065179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:22.283981 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc93ff-9f81-4c33-a6c6-e1a995065179 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.284364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc93ff-9f81-4c33-a6c6-e1a995065179 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.284641 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc93ff-9f81-4c33-a6c6-e1a995065179 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.295396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc93ff-9f81-4c33-a6c6-e1a995065179 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-94dc93ff-9f81-4c33-a6c6-e1a995065179 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:22.300224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:22.301369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:22.302466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:22.302466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:22.302466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:22.302466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:22.302466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:22.302466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:22.302466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:22.302466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:22.302466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:22.302466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:22.302466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:22.302466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:46:22.302466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 574/1149] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:22.307408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d605a3-0515-47ab-8476-b8f2bb160c1d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:22.405047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d605a3-0515-47ab-8476-b8f2bb160c1d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:22.437449 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902bf39-5c1a-4290-a4ee-4f8e1d2e37a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:22.438044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902bf39-5c1a-4290-a4ee-4f8e1d2e37a4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.438327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902bf39-5c1a-4290-a4ee-4f8e1d2e37a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.438602 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c902bf39-5c1a-4290-a4ee-4f8e1d2e37a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.439254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902bf39-5c1a-4290-a4ee-4f8e1d2e37a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:22.439557 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902bf39-5c1a-4290-a4ee-4f8e1d2e37a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:22.440950 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c902bf39-5c1a-4290-a4ee-4f8e1d2e37a4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:22.448730 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 576/1150] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:22.455071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc93ff-9f81-4c33-a6c6-e1a995065179 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:22.466689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:22.553318 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc93ff-9f81-4c33-a6c6-e1a995065179 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:22.586691 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413fd96-0b24-45d9-81a4-f350a0552d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:22.587311 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413fd96-0b24-45d9-81a4-f350a0552d66 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.587590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413fd96-0b24-45d9-81a4-f350a0552d66 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.587887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413fd96-0b24-45d9-81a4-f350a0552d66 None admin] PATH_INFO: `/v3/users/ccee017e80664ccda2b54e6de3f223e5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.588991 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413fd96-0b24-45d9-81a4-f350a0552d66 None admin] RBAC: Authorizing `identity:get_user(user_id=ccee017e80664ccda2b54e6de3f223e5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:22.590466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f413fd96-0b24-45d9-81a4-f350a0552d66 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:22.592774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 575/1151] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:46:22 2022] GET /identity/v3/users/ccee017e80664ccda2b54e6de3f223e5 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:22.598469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902bf39-5c1a-4290-a4ee-4f8e1d2e37a4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:22.632228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:22.698328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c902bf39-5c1a-4290-a4ee-4f8e1d2e37a4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:22.731269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff598f-71af-4446-b11f-bd72fd2dc962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:22.732073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff598f-71af-4446-b11f-bd72fd2dc962 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.732456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff598f-71af-4446-b11f-bd72fd2dc962 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.732735 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff598f-71af-4446-b11f-bd72fd2dc962 None admin] PATH_INFO: `/v3/projects/b65699c9f08a4497a810e9acb5966b92` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.733848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff598f-71af-4446-b11f-bd72fd2dc962 None admin] RBAC: Authorizing `identity:get_project(project_id=b65699c9f08a4497a810e9acb5966b92)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:22.735263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff598f-71af-4446-b11f-bd72fd2dc962 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:22.736946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 577/1152] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:46:22 2022] GET /identity/v3/projects/b65699c9f08a4497a810e9acb5966b92 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:22.742985 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413fd96-0b24-45d9-81a4-f350a0552d66 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:22.818536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:22.841093 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413fd96-0b24-45d9-81a4-f350a0552d66 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:22.875269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365a62a9-d8ec-412d-be82-cc455f62c9a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:22.875943 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a62a9-d8ec-412d-be82-cc455f62c9a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:22.876222 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a62a9-d8ec-412d-be82-cc455f62c9a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:22.876491 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a62a9-d8ec-412d-be82-cc455f62c9a9 None admin] PATH_INFO: `/v3/projects/b65699c9f08a4497a810e9acb5966b92/users/ccee017e80664ccda2b54e6de3f223e5/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:22.878319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a62a9-d8ec-412d-be82-cc455f62c9a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=b65699c9f08a4497a810e9acb5966b92, user_id=ccee017e80664ccda2b54e6de3f223e5, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:22.879757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365a62a9-d8ec-412d-be82-cc455f62c9a9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:22.933805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 576/1153] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:46:22 2022] PUT /identity/v3/projects/b65699c9f08a4497a810e9acb5966b92/users/ccee017e80664ccda2b54e6de3f223e5/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:46:25.058804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ae7f-3e4f-4972-884a-5f1c9fedb966 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:25.059316 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ae7f-3e4f-4972-884a-5f1c9fedb966 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:25.059433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0ae7f-3e4f-4972-884a-5f1c9fedb966 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:25.061478 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 578/1154] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:46:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:25.066702 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f371b-c73c-48c2-96a5-6e565456d141 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:25.067138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f371b-c73c-48c2-96a5-6e565456d141 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:25.067237 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f371b-c73c-48c2-96a5-6e565456d141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:25.086139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c72f371b-c73c-48c2-96a5-6e565456d141 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:25.087323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c72f371b-c73c-48c2-96a5-6e565456d141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:25.153075 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 577/1155] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:25.435327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0ae7f-3e4f-4972-884a-5f1c9fedb966 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:25.536759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0ae7f-3e4f-4972-884a-5f1c9fedb966 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:25.538258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:25.570768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378ac11-14f9-4237-8f49-cae932d65cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:25.571387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378ac11-14f9-4237-8f49-cae932d65cad None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:25.571689 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378ac11-14f9-4237-8f49-cae932d65cad None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:25.572044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2378ac11-14f9-4237-8f49-cae932d65cad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:25.582931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2378ac11-14f9-4237-8f49-cae932d65cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2378ac11-14f9-4237-8f49-cae932d65cad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:25.588216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:25.589332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:25.590532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:25.590532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:25.590532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:25.590532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:25.590532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:25.590532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:25.590532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:25.590532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:25.590532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:25.590532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:25.590532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:25.590532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:46:25.590532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 579/1156] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:25.596266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f371b-c73c-48c2-96a5-6e565456d141 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:25.670527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:25.705864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72f371b-c73c-48c2-96a5-6e565456d141 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:25.739870 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a8126-bb25-4b52-82b9-8c4e00d87352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:25.740470 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a8126-bb25-4b52-82b9-8c4e00d87352 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:25.740750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a8126-bb25-4b52-82b9-8c4e00d87352 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:25.741024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674a8126-bb25-4b52-82b9-8c4e00d87352 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:25.741681 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a8126-bb25-4b52-82b9-8c4e00d87352 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:25.741985 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a8126-bb25-4b52-82b9-8c4e00d87352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:25.743349 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674a8126-bb25-4b52-82b9-8c4e00d87352 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:25.751283 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 578/1157] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:25.757232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378ac11-14f9-4237-8f49-cae932d65cad None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:25.855663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2378ac11-14f9-4237-8f49-cae932d65cad None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:25.889228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a95b51-4e8f-499f-86ff-88ded478b7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:25.889855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a95b51-4e8f-499f-86ff-88ded478b7bc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:25.890138 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a95b51-4e8f-499f-86ff-88ded478b7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:25.890413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a95b51-4e8f-499f-86ff-88ded478b7bc None admin] PATH_INFO: `/v3/users/ccee017e80664ccda2b54e6de3f223e5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:25.891527 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a95b51-4e8f-499f-86ff-88ded478b7bc None admin] RBAC: Authorizing `identity:get_user(user_id=ccee017e80664ccda2b54e6de3f223e5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:25.892978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a95b51-4e8f-499f-86ff-88ded478b7bc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:25.894540 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 580/1158] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:46:25 2022] GET /identity/v3/users/ccee017e80664ccda2b54e6de3f223e5 => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:25.900430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a8126-bb25-4b52-82b9-8c4e00d87352 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:25.998464 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674a8126-bb25-4b52-82b9-8c4e00d87352 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:26.023783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:26.031947 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaff8e-e17d-4da9-9482-cb8116db4120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:26.032686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaff8e-e17d-4da9-9482-cb8116db4120 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:26.032981 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaff8e-e17d-4da9-9482-cb8116db4120 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:26.033257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeaff8e-e17d-4da9-9482-cb8116db4120 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:26.098429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeaff8e-e17d-4da9-9482-cb8116db4120 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:26.099892 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeaff8e-e17d-4da9-9482-cb8116db4120 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:26.194030 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 579/1159] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:46:26.200013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a95b51-4e8f-499f-86ff-88ded478b7bc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:26.230167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:26.298373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a95b51-4e8f-499f-86ff-88ded478b7bc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:26.330842 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e6fa12-1829-445b-91a7-a3bd69caca07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:26.331453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6fa12-1829-445b-91a7-a3bd69caca07 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:26.331756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6fa12-1829-445b-91a7-a3bd69caca07 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:26.332028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e6fa12-1829-445b-91a7-a3bd69caca07 None admin] PATH_INFO: `/v3/projects/b65699c9f08a4497a810e9acb5966b92` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:26.333112 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6fa12-1829-445b-91a7-a3bd69caca07 None admin] RBAC: Authorizing `identity:get_project(project_id=b65699c9f08a4497a810e9acb5966b92)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:26.334525 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e6fa12-1829-445b-91a7-a3bd69caca07 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:26.336139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 581/1160] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:46:26 2022] GET /identity/v3/projects/b65699c9f08a4497a810e9acb5966b92 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:26.342989 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaff8e-e17d-4da9-9482-cb8116db4120 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:26.370048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:26.441253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeaff8e-e17d-4da9-9482-cb8116db4120 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:26.474704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec3710d-fc10-4cc7-b74a-5127336b2166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:26.475298 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3710d-fc10-4cc7-b74a-5127336b2166 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:26.475578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3710d-fc10-4cc7-b74a-5127336b2166 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:26.475877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec3710d-fc10-4cc7-b74a-5127336b2166 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:26.476594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec3710d-fc10-4cc7-b74a-5127336b2166 None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=b65699c9f08a4497a810e9acb5966b92, user.id=ccee017e80664ccda2b54e6de3f223e5) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:26.476902 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec3710d-fc10-4cc7-b74a-5127336b2166 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:26.478315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec3710d-fc10-4cc7-b74a-5127336b2166 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:26.484194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 580/1161] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:26 2022] GET /identity/v3/role_assignments?user.id=ccee017e80664ccda2b54e6de3f223e5&scope.project.id=b65699c9f08a4497a810e9acb5966b92&role.id=07626899472b4e32adc0c71ce796a25d => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:28.671097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d412103-2714-4044-851c-0b28f574df86 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:28.671682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d412103-2714-4044-851c-0b28f574df86 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:28.671682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d412103-2714-4044-851c-0b28f574df86 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:28.673000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 582/1162] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:46:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:28.678328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1a563-fa33-451b-9b62-cfe2a542b0a4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:28.678751 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1a563-fa33-451b-9b62-cfe2a542b0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:28.678878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1a563-fa33-451b-9b62-cfe2a542b0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:28.697836 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3fe1a563-fa33-451b-9b62-cfe2a542b0a4 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:28.699005 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3fe1a563-fa33-451b-9b62-cfe2a542b0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:28.763243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 581/1163] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:46:29.027163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d412103-2714-4044-851c-0b28f574df86 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:29.121374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:29.126161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d412103-2714-4044-851c-0b28f574df86 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:29.159443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b0f83-4dc9-4ae1-8a81-66573a54e813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:29.160061 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b0f83-4dc9-4ae1-8a81-66573a54e813 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:29.160339 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b0f83-4dc9-4ae1-8a81-66573a54e813 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:29.160702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64b0f83-4dc9-4ae1-8a81-66573a54e813 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:29.225071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b0f83-4dc9-4ae1-8a81-66573a54e813 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:29.226493 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64b0f83-4dc9-4ae1-8a81-66573a54e813 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:29.319461 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:29.323938 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 583/1164] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:46:29.329746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1a563-fa33-451b-9b62-cfe2a542b0a4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:29.352946 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:29.426894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1a563-fa33-451b-9b62-cfe2a542b0a4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:29.460460 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f216df8-e1d9-4807-a533-73e0c08a28c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:29.461087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f216df8-e1d9-4807-a533-73e0c08a28c0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:29.461369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f216df8-e1d9-4807-a533-73e0c08a28c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:29.461644 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f216df8-e1d9-4807-a533-73e0c08a28c0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:29.468927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f216df8-e1d9-4807-a533-73e0c08a28c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:29.470374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f216df8-e1d9-4807-a533-73e0c08a28c0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3f216df8-e1d9-4807-a533-73e0c08a28c0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:29.482444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:29.483669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:29.484715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:29.484715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:29.484715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:29.484715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:29.484715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:29.484715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:29.484715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:29.484715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:29.484715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:29.484715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:29.484715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 26 17:46:29.484715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:46:29.484715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 26 17:46:29.484715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:46:29.484715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 582/1165] 10.222.0.67 () {56 vars in 1117 bytes} [Mon Sep 26 17:46:29 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:29.489520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b0f83-4dc9-4ae1-8a81-66573a54e813 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:29.586927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64b0f83-4dc9-4ae1-8a81-66573a54e813 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:29.621234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7fed5-d74a-4f14-aae6-4118ff553c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:29.621830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7fed5-d74a-4f14-aae6-4118ff553c38 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:29.622681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7fed5-d74a-4f14-aae6-4118ff553c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:29.622957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7fed5-d74a-4f14-aae6-4118ff553c38 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:29.623589 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7fed5-d74a-4f14-aae6-4118ff553c38 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:29.623918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7fed5-d74a-4f14-aae6-4118ff553c38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:29.625275 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7fed5-d74a-4f14-aae6-4118ff553c38 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:29.632196 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 584/1166] 10.222.0.67 () {56 vars in 1115 bytes} [Mon Sep 26 17:46:29 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:31.748298 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05889f2-2bf3-4cd3-8e95-b1648308ebe5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:31.748868 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05889f2-2bf3-4cd3-8e95-b1648308ebe5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:31.749020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05889f2-2bf3-4cd3-8e95-b1648308ebe5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:31.750296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 583/1167] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:46:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:31.755502 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708fbb8-9226-4523-97a3-2d9be2be7d6e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:31.755960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708fbb8-9226-4523-97a3-2d9be2be7d6e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:31.756087 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a708fbb8-9226-4523-97a3-2d9be2be7d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:31.774794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a708fbb8-9226-4523-97a3-2d9be2be7d6e None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:31.776007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a708fbb8-9226-4523-97a3-2d9be2be7d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:31.831520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:31.841861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 585/1168] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:32.107035 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05889f2-2bf3-4cd3-8e95-b1648308ebe5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:32.187333 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:32.205821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05889f2-2bf3-4cd3-8e95-b1648308ebe5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:32.239225 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87469450-7675-42de-a9f1-7a53a98317a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:32.239848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87469450-7675-42de-a9f1-7a53a98317a5 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:32.240125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87469450-7675-42de-a9f1-7a53a98317a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:32.240395 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87469450-7675-42de-a9f1-7a53a98317a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:32.241054 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87469450-7675-42de-a9f1-7a53a98317a5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:32.242413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87469450-7675-42de-a9f1-7a53a98317a5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:32.311730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 584/1169] 10.222.0.67 () {60 vars in 1138 bytes} [Mon Sep 26 17:46:32 2022] POST /identity/v3/domains => generated 267 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:34.534475 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5690d-c587-4dea-9833-cd80dd253066 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:34.535006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5690d-c587-4dea-9833-cd80dd253066 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:34.535006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5690d-c587-4dea-9833-cd80dd253066 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:34.536516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 586/1170] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:46:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:34.541643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c670e-b9ec-4b35-86ae-8783a4c8632e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:34.542060 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c670e-b9ec-4b35-86ae-8783a4c8632e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:34.542186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80c670e-b9ec-4b35-86ae-8783a4c8632e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:34.561051 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c80c670e-b9ec-4b35-86ae-8783a4c8632e None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:34.562221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c80c670e-b9ec-4b35-86ae-8783a4c8632e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:34.626611 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 585/1171] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:34.901437 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5690d-c587-4dea-9833-cd80dd253066 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:35.002890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5690d-c587-4dea-9833-cd80dd253066 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:35.037012 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f62bbb-e0f8-41ab-b1a8-c5ad40ecf474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:35.038066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f62bbb-e0f8-41ab-b1a8-c5ad40ecf474 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:35.038555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f62bbb-e0f8-41ab-b1a8-c5ad40ecf474 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:35.039025 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f62bbb-e0f8-41ab-b1a8-c5ad40ecf474 None admin] PATH_INFO: `/v3/domains/cc4eab2de1ae478bba16944698d1dd7f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:35.053898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f62bbb-e0f8-41ab-b1a8-c5ad40ecf474 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cc4eab2de1ae478bba16944698d1dd7f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:35.055345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f62bbb-e0f8-41ab-b1a8-c5ad40ecf474 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:35.057025 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 587/1172] 10.222.0.67 () {56 vars in 1183 bytes} [Mon Sep 26 17:46:34 2022] GET /identity/v3/domains/cc4eab2de1ae478bba16944698d1dd7f => generated 267 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:35.063092 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c670e-b9ec-4b35-86ae-8783a4c8632e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:35.132448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:35.162276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80c670e-b9ec-4b35-86ae-8783a4c8632e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:35.196535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1276bd-e3fc-4402-b601-0a56c068e34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:35.197138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1276bd-e3fc-4402-b601-0a56c068e34a None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:35.197415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1276bd-e3fc-4402-b601-0a56c068e34a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:35.197687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1276bd-e3fc-4402-b601-0a56c068e34a None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:35.198339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1276bd-e3fc-4402-b601-0a56c068e34a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:35.200400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1276bd-e3fc-4402-b601-0a56c068e34a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:35.271809 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 586/1173] 10.222.0.67 () {60 vars in 1141 bytes} [Mon Sep 26 17:46:35 2022] POST /identity/v3/projects => generated 361 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:37.443905 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb827b8-6a93-4577-8ba0-6134728de1ab None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:37.444460 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb827b8-6a93-4577-8ba0-6134728de1ab None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:37.444460 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb827b8-6a93-4577-8ba0-6134728de1ab None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:37.445762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 588/1174] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:46:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:37.451800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810835e3-d2e8-45c3-b5c1-46a23297c7fc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:37.452221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810835e3-d2e8-45c3-b5c1-46a23297c7fc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:37.452364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810835e3-d2e8-45c3-b5c1-46a23297c7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:37.471798 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-810835e3-d2e8-45c3-b5c1-46a23297c7fc None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:37.472991 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-810835e3-d2e8-45c3-b5c1-46a23297c7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:37.538321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 587/1175] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:37.813867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb827b8-6a93-4577-8ba0-6134728de1ab None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:37.867775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:37.914929 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb827b8-6a93-4577-8ba0-6134728de1ab None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:37.948331 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fa649-9bfe-414f-bbbe-a0aacc55da70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:37.948945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fa649-9bfe-414f-bbbe-a0aacc55da70 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:37.949228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fa649-9bfe-414f-bbbe-a0aacc55da70 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:37.949504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fa649-9bfe-414f-bbbe-a0aacc55da70 None admin] PATH_INFO: `/v3/domains/cc4eab2de1ae478bba16944698d1dd7f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:37.950988 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fa649-9bfe-414f-bbbe-a0aacc55da70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cc4eab2de1ae478bba16944698d1dd7f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:37.952433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fa649-9bfe-414f-bbbe-a0aacc55da70 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:37.953981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 589/1176] 10.222.0.67 () {56 vars in 1183 bytes} [Mon Sep 26 17:46:37 2022] GET /identity/v3/domains/cc4eab2de1ae478bba16944698d1dd7f => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:37.959749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810835e3-d2e8-45c3-b5c1-46a23297c7fc None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:38.014024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:38.058367 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810835e3-d2e8-45c3-b5c1-46a23297c7fc None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:38.092509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc061749-32f6-4dca-88f6-2d78a1a2507a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:38.093024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc061749-32f6-4dca-88f6-2d78a1a2507a None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:38.093299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc061749-32f6-4dca-88f6-2d78a1a2507a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:38.093564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc061749-32f6-4dca-88f6-2d78a1a2507a None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:38.094208 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc061749-32f6-4dca-88f6-2d78a1a2507a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:38.095631 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc061749-32f6-4dca-88f6-2d78a1a2507a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:38.190706 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 588/1177] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:46:37 2022] POST /identity/v3/users => generated 311 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:46:40.399908 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24fe62-f865-42d5-b7a4-bf65cd9ccd07 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:40.400457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24fe62-f865-42d5-b7a4-bf65cd9ccd07 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:40.400560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24fe62-f865-42d5-b7a4-bf65cd9ccd07 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:40.401861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 590/1178] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:46:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:40.407548 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15773ba-feea-43ce-a5ef-af17d6a3d8d5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:40.407985 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15773ba-feea-43ce-a5ef-af17d6a3d8d5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:40.408129 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15773ba-feea-43ce-a5ef-af17d6a3d8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:40.426710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f15773ba-feea-43ce-a5ef-af17d6a3d8d5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:40.427926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f15773ba-feea-43ce-a5ef-af17d6a3d8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:40.493253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 589/1179] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:40.772155 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24fe62-f865-42d5-b7a4-bf65cd9ccd07 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:40.870847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c24fe62-f865-42d5-b7a4-bf65cd9ccd07 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:40.890396 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:40.904326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7b593-4b70-444d-8514-bb37ea1f8b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:40.904946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7b593-4b70-444d-8514-bb37ea1f8b18 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:40.905225 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7b593-4b70-444d-8514-bb37ea1f8b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:40.905494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7b593-4b70-444d-8514-bb37ea1f8b18 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:40.916011 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7b593-4b70-444d-8514-bb37ea1f8b18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-84e7b593-4b70-444d-8514-bb37ea1f8b18 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:40.921958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:40.923031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:40.924250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:40.924250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:40.924250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:40.924250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:40.924250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:40.924250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:40.924250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:40.924250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:40.924250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:40.924250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:40.924250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:40.924250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:46:40.924250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 591/1180] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:40.929009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15773ba-feea-43ce-a5ef-af17d6a3d8d5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:40.969841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:41.030265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15773ba-feea-43ce-a5ef-af17d6a3d8d5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:41.069919 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d85c4-9ea6-4d35-bf8b-69b6b82b7d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:41.070524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d85c4-9ea6-4d35-bf8b-69b6b82b7d89 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:41.070803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d85c4-9ea6-4d35-bf8b-69b6b82b7d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:41.071104 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45d85c4-9ea6-4d35-bf8b-69b6b82b7d89 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:41.071813 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d85c4-9ea6-4d35-bf8b-69b6b82b7d89 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:41.072118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d85c4-9ea6-4d35-bf8b-69b6b82b7d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:41.073489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45d85c4-9ea6-4d35-bf8b-69b6b82b7d89 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:41.082077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 590/1181] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:41.088013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7b593-4b70-444d-8514-bb37ea1f8b18 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:41.187263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7b593-4b70-444d-8514-bb37ea1f8b18 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:41.220364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbf240-f531-4865-9d24-fa170287f730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:41.220979 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbf240-f531-4865-9d24-fa170287f730 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:41.221263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbf240-f531-4865-9d24-fa170287f730 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:41.221538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbf240-f531-4865-9d24-fa170287f730 None admin] PATH_INFO: `/v3/users/4f500bb275fb44a8a44ca5f5a94a35f2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:41.234222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbf240-f531-4865-9d24-fa170287f730 None admin] RBAC: Authorizing `identity:get_user(user_id=4f500bb275fb44a8a44ca5f5a94a35f2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:41.235664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbf240-f531-4865-9d24-fa170287f730 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:41.237254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 592/1182] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:46:41 2022] GET /identity/v3/users/4f500bb275fb44a8a44ca5f5a94a35f2 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:41.242914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d85c4-9ea6-4d35-bf8b-69b6b82b7d89 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:41.342012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45d85c4-9ea6-4d35-bf8b-69b6b82b7d89 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:41.343458 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:41.380075 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894c6fe-4ee0-4b53-8247-a428db138797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:41.380686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894c6fe-4ee0-4b53-8247-a428db138797 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:41.380966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894c6fe-4ee0-4b53-8247-a428db138797 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:41.381248 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894c6fe-4ee0-4b53-8247-a428db138797 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:41.448246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894c6fe-4ee0-4b53-8247-a428db138797 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:41.450409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894c6fe-4ee0-4b53-8247-a428db138797 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:41.547249 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 591/1183] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:46:41.553254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbf240-f531-4865-9d24-fa170287f730 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:41.593487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:41.654211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbf240-f531-4865-9d24-fa170287f730 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:41.687517 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c74bd2-ccd4-46ea-b59e-0438ff52803e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:41.688165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c74bd2-ccd4-46ea-b59e-0438ff52803e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:41.688445 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c74bd2-ccd4-46ea-b59e-0438ff52803e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:41.688713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c74bd2-ccd4-46ea-b59e-0438ff52803e None admin] PATH_INFO: `/v3/projects/e1c154cbb8d741199c57c68f2a8cbff9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:41.689808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c74bd2-ccd4-46ea-b59e-0438ff52803e None admin] RBAC: Authorizing `identity:get_project(project_id=e1c154cbb8d741199c57c68f2a8cbff9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:41.691228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c74bd2-ccd4-46ea-b59e-0438ff52803e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:41.692865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 593/1184] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:46:41 2022] GET /identity/v3/projects/e1c154cbb8d741199c57c68f2a8cbff9 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:41.698964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894c6fe-4ee0-4b53-8247-a428db138797 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:41.701881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:41.797210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894c6fe-4ee0-4b53-8247-a428db138797 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:41.830256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ab802-1f9f-4385-b573-cb0ffe37d6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:41.830849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ab802-1f9f-4385-b573-cb0ffe37d6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:41.831127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ab802-1f9f-4385-b573-cb0ffe37d6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:41.831395 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ab802-1f9f-4385-b573-cb0ffe37d6f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:41.832268 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ab802-1f9f-4385-b573-cb0ffe37d6f1 None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=e1c154cbb8d741199c57c68f2a8cbff9, user.id=4f500bb275fb44a8a44ca5f5a94a35f2) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:41.832571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ab802-1f9f-4385-b573-cb0ffe37d6f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:41.833975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ab802-1f9f-4385-b573-cb0ffe37d6f1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:41.839109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 592/1185] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:41 2022] GET /identity/v3/role_assignments?user.id=4f500bb275fb44a8a44ca5f5a94a35f2&scope.project.id=e1c154cbb8d741199c57c68f2a8cbff9&role.id=07626899472b4e32adc0c71ce796a25d => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:43.951670 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd8e8f-e1b6-4f62-af20-e2a9502c3831 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:43.952198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd8e8f-e1b6-4f62-af20-e2a9502c3831 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:43.952309 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd8e8f-e1b6-4f62-af20-e2a9502c3831 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:43.953561 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 594/1186] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:46:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:43.958605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352ba52-336d-40f3-a7a2-3c7c6a49a694 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:43.959006 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352ba52-336d-40f3-a7a2-3c7c6a49a694 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:43.959160 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5352ba52-336d-40f3-a7a2-3c7c6a49a694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:43.977605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5352ba52-336d-40f3-a7a2-3c7c6a49a694 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:43.978811 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5352ba52-336d-40f3-a7a2-3c7c6a49a694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:44.037980 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:44.046341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 593/1187] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:44.311205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd8e8f-e1b6-4f62-af20-e2a9502c3831 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:44.409608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd8e8f-e1b6-4f62-af20-e2a9502c3831 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:44.416571 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:44.442806 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e79a4-5998-4a3f-a07d-80fea207f82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:44.443421 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e79a4-5998-4a3f-a07d-80fea207f82d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:44.443728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e79a4-5998-4a3f-a07d-80fea207f82d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:44.444005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5e79a4-5998-4a3f-a07d-80fea207f82d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:44.454329 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5e79a4-5998-4a3f-a07d-80fea207f82d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-9c5e79a4-5998-4a3f-a07d-80fea207f82d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:44.459136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:44.460267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:44.461318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:44.461318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:44.461318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:44.461318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:44.461318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:44.461318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:44.461318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:44.461318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:44.461318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:44.461318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:44.461318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:44.461318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:46:44.461318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 595/1188] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:44.466633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352ba52-336d-40f3-a7a2-3c7c6a49a694 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:44.564508 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5352ba52-336d-40f3-a7a2-3c7c6a49a694 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:44.598069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdc429-eb3a-4704-9adf-4a2a4c131199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:44.598665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdc429-eb3a-4704-9adf-4a2a4c131199 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:44.598945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdc429-eb3a-4704-9adf-4a2a4c131199 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:44.599217 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdc429-eb3a-4704-9adf-4a2a4c131199 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:44.599895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdc429-eb3a-4704-9adf-4a2a4c131199 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:44.600197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdc429-eb3a-4704-9adf-4a2a4c131199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:44.601560 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdc429-eb3a-4704-9adf-4a2a4c131199 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:44.609706 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 594/1189] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:44.615380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e79a4-5998-4a3f-a07d-80fea207f82d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:44.714832 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5e79a4-5998-4a3f-a07d-80fea207f82d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:44.747688 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256692a-7e17-4067-9f38-a63fcc9ca4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:44.748312 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256692a-7e17-4067-9f38-a63fcc9ca4d7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:44.748594 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256692a-7e17-4067-9f38-a63fcc9ca4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:44.748867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8256692a-7e17-4067-9f38-a63fcc9ca4d7 None admin] PATH_INFO: `/v3/users/4f500bb275fb44a8a44ca5f5a94a35f2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:44.749967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256692a-7e17-4067-9f38-a63fcc9ca4d7 None admin] RBAC: Authorizing `identity:get_user(user_id=4f500bb275fb44a8a44ca5f5a94a35f2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:44.751382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8256692a-7e17-4067-9f38-a63fcc9ca4d7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:44.752958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 596/1190] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:46:44 2022] GET /identity/v3/users/4f500bb275fb44a8a44ca5f5a94a35f2 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:46:44.758629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdc429-eb3a-4704-9adf-4a2a4c131199 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:44.817497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:44.859775 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdc429-eb3a-4704-9adf-4a2a4c131199 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:44.893310 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac0c492-4f92-4d13-ad01-bcbef1ddf146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:44.893935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0c492-4f92-4d13-ad01-bcbef1ddf146 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:44.894216 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0c492-4f92-4d13-ad01-bcbef1ddf146 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:44.894492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac0c492-4f92-4d13-ad01-bcbef1ddf146 None admin] PATH_INFO: `/v3/projects/e1c154cbb8d741199c57c68f2a8cbff9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:44.895597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0c492-4f92-4d13-ad01-bcbef1ddf146 None admin] RBAC: Authorizing `identity:get_project(project_id=e1c154cbb8d741199c57c68f2a8cbff9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:44.897060 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac0c492-4f92-4d13-ad01-bcbef1ddf146 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:44.898670 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 595/1191] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:46:44 2022] GET /identity/v3/projects/e1c154cbb8d741199c57c68f2a8cbff9 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:44.904856 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256692a-7e17-4067-9f38-a63fcc9ca4d7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:44.940144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:45.005513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8256692a-7e17-4067-9f38-a63fcc9ca4d7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:45.038048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b49a20-fa64-4f02-884e-c346027f0378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:45.038697 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b49a20-fa64-4f02-884e-c346027f0378 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:45.038976 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b49a20-fa64-4f02-884e-c346027f0378 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:45.039250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b49a20-fa64-4f02-884e-c346027f0378 None admin] PATH_INFO: `/v3/projects/e1c154cbb8d741199c57c68f2a8cbff9/users/4f500bb275fb44a8a44ca5f5a94a35f2/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:45.041124 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b49a20-fa64-4f02-884e-c346027f0378 None admin] RBAC: Authorizing `identity:create_grant(project_id=e1c154cbb8d741199c57c68f2a8cbff9, user_id=4f500bb275fb44a8a44ca5f5a94a35f2, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:45.042876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b49a20-fa64-4f02-884e-c346027f0378 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:45.127380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 597/1192] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:46:44 2022] PUT /identity/v3/projects/e1c154cbb8d741199c57c68f2a8cbff9/users/4f500bb275fb44a8a44ca5f5a94a35f2/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:46:47.253942 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13273efc-bd8b-49c1-b26e-718f440b8331 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:47.254452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13273efc-bd8b-49c1-b26e-718f440b8331 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:47.254452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13273efc-bd8b-49c1-b26e-718f440b8331 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:47.255771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 596/1193] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:46:47.261062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8c774-9368-4f33-af93-547d1cff4663 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:47.261429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8c774-9368-4f33-af93-547d1cff4663 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:47.261613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8c774-9368-4f33-af93-547d1cff4663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:47.279847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-deb8c774-9368-4f33-af93-547d1cff4663 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:46:47.281017 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-deb8c774-9368-4f33-af93-547d1cff4663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:46:47.345627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 598/1194] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:46:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:46:47.614778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13273efc-bd8b-49c1-b26e-718f440b8331 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:47.715805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13273efc-bd8b-49c1-b26e-718f440b8331 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:47.739710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:47.748605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed5b76-f7dd-4578-936f-93435389d4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:47.749224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed5b76-f7dd-4578-936f-93435389d4b2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:47.752441 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed5b76-f7dd-4578-936f-93435389d4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:47.752441 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ed5b76-f7dd-4578-936f-93435389d4b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:47.760290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ed5b76-f7dd-4578-936f-93435389d4b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e0ed5b76-f7dd-4578-936f-93435389d4b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:46:47.766363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:46:47.767281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:46:47.768392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:46:47.768392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:46:47.768392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:46:47.768392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:46:47.768392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:46:47.768392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:46:47.768392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:46:47.768392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:46:47.768392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:46:47.768392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:46:47.768392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:46:47.768392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:46:47.768392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 597/1195] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:46:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:46:47.773731 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8c774-9368-4f33-af93-547d1cff4663 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:47.797665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:47.871718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb8c774-9368-4f33-af93-547d1cff4663 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:47.905583 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300b35d-9f8d-4e25-966c-ff2f87dec56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:47.906180 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300b35d-9f8d-4e25-966c-ff2f87dec56e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:47.906461 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300b35d-9f8d-4e25-966c-ff2f87dec56e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:47.906732 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1300b35d-9f8d-4e25-966c-ff2f87dec56e None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:47.907417 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300b35d-9f8d-4e25-966c-ff2f87dec56e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:47.907748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300b35d-9f8d-4e25-966c-ff2f87dec56e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:47.909111 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1300b35d-9f8d-4e25-966c-ff2f87dec56e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:47.916872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 599/1196] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:46:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:47.922823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed5b76-f7dd-4578-936f-93435389d4b2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:48.023177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ed5b76-f7dd-4578-936f-93435389d4b2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:48.055133 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd216b-d25b-4235-9073-c9985b630060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:48.055787 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd216b-d25b-4235-9073-c9985b630060 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:48.056067 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd216b-d25b-4235-9073-c9985b630060 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:48.056338 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dd216b-d25b-4235-9073-c9985b630060 None admin] PATH_INFO: `/v3/users/4f500bb275fb44a8a44ca5f5a94a35f2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:48.057431 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd216b-d25b-4235-9073-c9985b630060 None admin] RBAC: Authorizing `identity:get_user(user_id=4f500bb275fb44a8a44ca5f5a94a35f2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:48.059676 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dd216b-d25b-4235-9073-c9985b630060 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:48.061126 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 598/1197] 10.222.0.67 () {56 vars in 1177 bytes} [Mon Sep 26 17:46:47 2022] GET /identity/v3/users/4f500bb275fb44a8a44ca5f5a94a35f2 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:48.066919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300b35d-9f8d-4e25-966c-ff2f87dec56e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:48.150567 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:48.164913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1300b35d-9f8d-4e25-966c-ff2f87dec56e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:48.198820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc906c-e343-4846-815e-3832409326f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:48.199676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc906c-e343-4846-815e-3832409326f8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:48.200164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc906c-e343-4846-815e-3832409326f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:48.200434 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc906c-e343-4846-815e-3832409326f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:48.265784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc906c-e343-4846-815e-3832409326f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:48.267222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc906c-e343-4846-815e-3832409326f8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:48.347180 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:48.362688 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 600/1198] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:46:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:46:48.368845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd216b-d25b-4235-9073-c9985b630060 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:48.423484 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:48.467943 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dd216b-d25b-4235-9073-c9985b630060 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:48.500860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b745a3b-a959-41b5-9925-c3854bae7a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:48.501494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b745a3b-a959-41b5-9925-c3854bae7a90 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:48.501778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b745a3b-a959-41b5-9925-c3854bae7a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:48.502055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b745a3b-a959-41b5-9925-c3854bae7a90 None admin] PATH_INFO: `/v3/projects/e1c154cbb8d741199c57c68f2a8cbff9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:48.503169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b745a3b-a959-41b5-9925-c3854bae7a90 None admin] RBAC: Authorizing `identity:get_project(project_id=e1c154cbb8d741199c57c68f2a8cbff9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:48.504632 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b745a3b-a959-41b5-9925-c3854bae7a90 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:48.506257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 599/1199] 10.222.0.67 () {56 vars in 1186 bytes} [Mon Sep 26 17:46:48 2022] GET /identity/v3/projects/e1c154cbb8d741199c57c68f2a8cbff9 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:48.513289 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc906c-e343-4846-815e-3832409326f8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:46:48.610066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc906c-e343-4846-815e-3832409326f8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:46:48.644017 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3bfdb8-bcce-4839-9238-c792e3f7804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:46:48.644612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bfdb8-bcce-4839-9238-c792e3f7804d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:46:48.644892 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bfdb8-bcce-4839-9238-c792e3f7804d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:46:48.645164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bfdb8-bcce-4839-9238-c792e3f7804d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:46:48.645874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3bfdb8-bcce-4839-9238-c792e3f7804d None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=e1c154cbb8d741199c57c68f2a8cbff9, user.id=4f500bb275fb44a8a44ca5f5a94a35f2) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:46:48.646183 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3bfdb8-bcce-4839-9238-c792e3f7804d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:46:48.647593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3bfdb8-bcce-4839-9238-c792e3f7804d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:46:48.653357 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 601/1200] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:46:48 2022] GET /identity/v3/role_assignments?user.id=4f500bb275fb44a8a44ca5f5a94a35f2&scope.project.id=e1c154cbb8d741199c57c68f2a8cbff9&role.id=07626899472b4e32adc0c71ce796a25d => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:46:58.496075 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=39 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:46:58.650619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=75 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:47:59.723399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f8c42-dfc4-4626-979e-9ca286fbbfce None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:59.724014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f8c42-dfc4-4626-979e-9ca286fbbfce None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:59.724135 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f8c42-dfc4-4626-979e-9ca286fbbfce None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:59.725529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 600/1201] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:47:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:47:59.731042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba53c2-3aee-476e-8f7e-c16adffd311d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:47:59.731484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba53c2-3aee-476e-8f7e-c16adffd311d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:47:59.731607 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba53c2-3aee-476e-8f7e-c16adffd311d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:47:59.750840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ddba53c2-3aee-476e-8f7e-c16adffd311d None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:47:59.752042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ddba53c2-3aee-476e-8f7e-c16adffd311d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:47:59.817972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 602/1202] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:47:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:48:00.087354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f8c42-dfc4-4626-979e-9ca286fbbfce None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:00.195175 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f8c42-dfc4-4626-979e-9ca286fbbfce None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:00.228393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34b070-7380-465d-b834-fc5e8da822ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:00.229013 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34b070-7380-465d-b834-fc5e8da822ba None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:00.229290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34b070-7380-465d-b834-fc5e8da822ba None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:00.229557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34b070-7380-465d-b834-fc5e8da822ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:00.240023 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34b070-7380-465d-b834-fc5e8da822ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:00.241390 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de34b070-7380-465d-b834-fc5e8da822ba None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-de34b070-7380-465d-b834-fc5e8da822ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:48:00.255930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:00.263439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:48:00.265306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:48:00.265306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:00.265306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:00.265306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 601/1203] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:00.265553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba53c2-3aee-476e-8f7e-c16adffd311d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:00.362838 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddba53c2-3aee-476e-8f7e-c16adffd311d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:00.391751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:00.397633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2bd53-f050-4844-974c-3798ea982b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:00.398242 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2bd53-f050-4844-974c-3798ea982b1e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:00.398600 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2bd53-f050-4844-974c-3798ea982b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:00.398876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2bd53-f050-4844-974c-3798ea982b1e None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:00.399515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2bd53-f050-4844-974c-3798ea982b1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:00.399856 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2bd53-f050-4844-974c-3798ea982b1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:00.401228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2bd53-f050-4844-974c-3798ea982b1e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:00.412977 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 603/1204] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:48:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:00.418895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34b070-7380-465d-b834-fc5e8da822ba None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:00.474224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:00.522621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34b070-7380-465d-b834-fc5e8da822ba None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:00.556397 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be97db-1d55-47b5-88b2-c56379ac8726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:00.556999 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be97db-1d55-47b5-88b2-c56379ac8726 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:00.557276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be97db-1d55-47b5-88b2-c56379ac8726 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:00.557552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be97db-1d55-47b5-88b2-c56379ac8726 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:00.558186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be97db-1d55-47b5-88b2-c56379ac8726 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:00.559609 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be97db-1d55-47b5-88b2-c56379ac8726 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:00.703448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 602/1205] 10.222.0.67 () {60 vars in 1132 bytes} [Mon Sep 26 17:48:00 2022] POST /identity/v3/users => generated 251 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:48:02.892185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1165ed-df55-4e34-8c28-3a8b6ea17cdc None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:02.892740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1165ed-df55-4e34-8c28-3a8b6ea17cdc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:02.892740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1165ed-df55-4e34-8c28-3a8b6ea17cdc None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:02.894118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 604/1206] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:48:02.899482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8d364-19ec-44c8-9463-4972a961c4be None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:02.899936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8d364-19ec-44c8-9463-4972a961c4be None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:02.900063 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a8d364-19ec-44c8-9463-4972a961c4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:02.919125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c5a8d364-19ec-44c8-9463-4972a961c4be None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:48:02.920332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c5a8d364-19ec-44c8-9463-4972a961c4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:48:02.990398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 603/1207] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:48:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:48:03.260196 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1165ed-df55-4e34-8c28-3a8b6ea17cdc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:03.363673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1165ed-df55-4e34-8c28-3a8b6ea17cdc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:03.396364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b661b4d-6380-4c8e-aa59-c89a581e4bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:03.396983 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b661b4d-6380-4c8e-aa59-c89a581e4bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:03.397262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b661b4d-6380-4c8e-aa59-c89a581e4bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:03.397534 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b661b4d-6380-4c8e-aa59-c89a581e4bf4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:03.408604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b661b4d-6380-4c8e-aa59-c89a581e4bf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4b661b4d-6380-4c8e-aa59-c89a581e4bf4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:48:03.413368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:03.414422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:03.415368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:03.415368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:03.415368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:03.415368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:03.415368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:48:03.415368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:48:03.415368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:48:03.415368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:48:03.415368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:48:03.415368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:48:03.415368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:48:03.415368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:03.415368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 605/1208] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:48:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:03.421174 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8d364-19ec-44c8-9463-4972a961c4be None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:03.461017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:03.522497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a8d364-19ec-44c8-9463-4972a961c4be None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:03.557613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61862cf-0383-4edd-b77a-71f2f845933c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:03.558223 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61862cf-0383-4edd-b77a-71f2f845933c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:03.558501 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61862cf-0383-4edd-b77a-71f2f845933c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:03.558774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61862cf-0383-4edd-b77a-71f2f845933c None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:03.559433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61862cf-0383-4edd-b77a-71f2f845933c None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:03.559768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61862cf-0383-4edd-b77a-71f2f845933c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:03.561255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61862cf-0383-4edd-b77a-71f2f845933c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:03.569595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 604/1209] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:48:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:03.575700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b661b4d-6380-4c8e-aa59-c89a581e4bf4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:03.627728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:03.677039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b661b4d-6380-4c8e-aa59-c89a581e4bf4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:03.709904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20805c-3124-4e93-b5b6-ffb03339b7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:03.710529 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20805c-3124-4e93-b5b6-ffb03339b7e2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:03.710808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20805c-3124-4e93-b5b6-ffb03339b7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:03.711080 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20805c-3124-4e93-b5b6-ffb03339b7e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:03.722150 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20805c-3124-4e93-b5b6-ffb03339b7e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:03.723579 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20805c-3124-4e93-b5b6-ffb03339b7e2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-df20805c-3124-4e93-b5b6-ffb03339b7e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:48:03.738658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:03.739821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:48:03.741128 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:48:03.741128 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:03.741128 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:03.741128 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 606/1210] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:03.745530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61862cf-0383-4edd-b77a-71f2f845933c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:03.845909 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:03.849063 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61862cf-0383-4edd-b77a-71f2f845933c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:03.882142 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c241d-b553-4398-8744-a75614dc40b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:03.882741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c241d-b553-4398-8744-a75614dc40b5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:03.883013 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c241d-b553-4398-8744-a75614dc40b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:03.883278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20c241d-b553-4398-8744-a75614dc40b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:03.883936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c241d-b553-4398-8744-a75614dc40b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:03.884234 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c241d-b553-4398-8744-a75614dc40b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:03.885587 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20c241d-b553-4398-8744-a75614dc40b5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:03.897315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 605/1211] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:03.903172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20805c-3124-4e93-b5b6-ffb03339b7e2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:04.004033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20805c-3124-4e93-b5b6-ffb03339b7e2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:04.005575 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:04.037463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac590b1-4471-485a-921d-291a969b4cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:04.038082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac590b1-4471-485a-921d-291a969b4cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:04.038357 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac590b1-4471-485a-921d-291a969b4cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:04.038625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac590b1-4471-485a-921d-291a969b4cd0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:04.046592 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac590b1-4471-485a-921d-291a969b4cd0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:04.048147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac590b1-4471-485a-921d-291a969b4cd0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8ac590b1-4471-485a-921d-291a969b4cd0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:48:04.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:04.061661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:04.062719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:04.062719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:04.062719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:48:04.062719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:48:04.062719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:48:04.062719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:48:04.062719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:48:04.062719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:48:04.062719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:48:04.062719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:04.062719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 607/1212] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:04.066272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c241d-b553-4398-8744-a75614dc40b5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:04.166065 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20c241d-b553-4398-8744-a75614dc40b5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:04.199693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262f777-14eb-45ac-b7ca-ffb36a21b6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:04.200295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262f777-14eb-45ac-b7ca-ffb36a21b6ba None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:04.200570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262f777-14eb-45ac-b7ca-ffb36a21b6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:04.200837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7262f777-14eb-45ac-b7ca-ffb36a21b6ba None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:04.202067 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262f777-14eb-45ac-b7ca-ffb36a21b6ba None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:04.202387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262f777-14eb-45ac-b7ca-ffb36a21b6ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:04.203832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7262f777-14eb-45ac-b7ca-ffb36a21b6ba None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:04.221275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 606/1213] 10.222.0.67 () {56 vars in 1143 bytes} [Mon Sep 26 17:48:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:04.227133 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac590b1-4471-485a-921d-291a969b4cd0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:04.326263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac590b1-4471-485a-921d-291a969b4cd0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:04.359737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837b305-07cc-45b2-9ec3-46718be0826b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:04.360333 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837b305-07cc-45b2-9ec3-46718be0826b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:04.360722 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837b305-07cc-45b2-9ec3-46718be0826b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:04.361581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837b305-07cc-45b2-9ec3-46718be0826b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:04.390429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:04.430722 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837b305-07cc-45b2-9ec3-46718be0826b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:04.430722 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837b305-07cc-45b2-9ec3-46718be0826b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:04.528359 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 608/1214] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:48:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:48:04.535073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262f777-14eb-45ac-b7ca-ffb36a21b6ba None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:04.553923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:04.631848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7262f777-14eb-45ac-b7ca-ffb36a21b6ba None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:04.666696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb957a-089f-4f71-b06b-ab167373a4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:04.667323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb957a-089f-4f71-b06b-ab167373a4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:04.667598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb957a-089f-4f71-b06b-ab167373a4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:04.667895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eb957a-089f-4f71-b06b-ab167373a4a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:04.677913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb957a-089f-4f71-b06b-ab167373a4a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:04.679462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eb957a-089f-4f71-b06b-ab167373a4a3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-91eb957a-089f-4f71-b06b-ab167373a4a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:48:04.694490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:04.695446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:48:04.696764 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:48:04.696764 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:04.696764 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:04.696764 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 607/1215] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:04.701378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837b305-07cc-45b2-9ec3-46718be0826b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:04.761854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:04.799855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837b305-07cc-45b2-9ec3-46718be0826b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:04.832820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf44a0e-5a21-468e-82e8-7099c923da49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:04.833415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf44a0e-5a21-468e-82e8-7099c923da49 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:04.833696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf44a0e-5a21-468e-82e8-7099c923da49 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:04.833969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf44a0e-5a21-468e-82e8-7099c923da49 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:04.834599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf44a0e-5a21-468e-82e8-7099c923da49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:04.834901 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf44a0e-5a21-468e-82e8-7099c923da49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:04.836284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf44a0e-5a21-468e-82e8-7099c923da49 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:04.847829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 609/1216] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:04.853696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb957a-089f-4f71-b06b-ab167373a4a3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:04.923220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:04.954962 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eb957a-089f-4f71-b06b-ab167373a4a3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:04.988746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1c91d-8328-480b-a5f9-103872243e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:04.989381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1c91d-8328-480b-a5f9-103872243e9a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:04.989656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1c91d-8328-480b-a5f9-103872243e9a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:04.989922 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f1c91d-8328-480b-a5f9-103872243e9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:04.997245 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1c91d-8328-480b-a5f9-103872243e9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:04.999402 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f1c91d-8328-480b-a5f9-103872243e9a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:05.012246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-01f1c91d-8328-480b-a5f9-103872243e9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:48:05.012246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:05.012246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:05.012246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:05.012246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:05.012246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:05.012246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:05.012246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:05.012246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:05.012246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:05.012246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:05.012246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:05.012246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:48:05.012246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:48:05.012246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:05.013296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:05.024738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:48:05.024738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:48:05.024738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:48:05.024738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:48:05.024738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:48:05.024738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:48:05.024738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:48:05.024738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:05.024738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 608/1217] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:48:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:05.025439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf44a0e-5a21-468e-82e8-7099c923da49 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:05.119194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf44a0e-5a21-468e-82e8-7099c923da49 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:05.142462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:05.153660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e9f4ca-a11d-403c-aa00-ea2baab59003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:05.154267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9f4ca-a11d-403c-aa00-ea2baab59003 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:05.154542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9f4ca-a11d-403c-aa00-ea2baab59003 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:05.154861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e9f4ca-a11d-403c-aa00-ea2baab59003 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:05.155532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9f4ca-a11d-403c-aa00-ea2baab59003 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:05.155855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9f4ca-a11d-403c-aa00-ea2baab59003 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:05.157271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e9f4ca-a11d-403c-aa00-ea2baab59003 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:05.169029 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 610/1218] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:48:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:48:05.175854 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1c91d-8328-480b-a5f9-103872243e9a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:05.282805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f1c91d-8328-480b-a5f9-103872243e9a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:05.311001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:05.316730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1774882d-2969-4f8f-a351-0c39439b86ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:05.317335 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774882d-2969-4f8f-a351-0c39439b86ea None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:05.317608 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774882d-2969-4f8f-a351-0c39439b86ea None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:05.317877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1774882d-2969-4f8f-a351-0c39439b86ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:05.318587 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774882d-2969-4f8f-a351-0c39439b86ea None admin] RBAC: Adding query filter params (role.id=90322db354f94f03b32ec088922baf2e, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=22a512e0a64e420abf15b2150ccc2b32) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:05.318891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774882d-2969-4f8f-a351-0c39439b86ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:05.320381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1774882d-2969-4f8f-a351-0c39439b86ea None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:05.325627 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 609/1219] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:48:05 2022] GET /identity/v3/role_assignments?user.id=22a512e0a64e420abf15b2150ccc2b32&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=90322db354f94f03b32ec088922baf2e => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:07.471284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e3050-be46-4920-b955-e25b31f0d22e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:07.471845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e3050-be46-4920-b955-e25b31f0d22e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:07.471845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e3050-be46-4920-b955-e25b31f0d22e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:07.473201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 611/1220] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:48:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:48:07.478466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44585ab5-7398-466e-ab8a-c10f797888c0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:07.482439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44585ab5-7398-466e-ab8a-c10f797888c0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:07.482439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44585ab5-7398-466e-ab8a-c10f797888c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:07.497394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-44585ab5-7398-466e-ab8a-c10f797888c0 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:48:07.499157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-44585ab5-7398-466e-ab8a-c10f797888c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:48:07.566901 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 610/1221] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:48:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:48:07.845672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e3050-be46-4920-b955-e25b31f0d22e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:07.947245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e3050-be46-4920-b955-e25b31f0d22e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:07.983074 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557adbcb-332d-4d0d-9794-4b6e0fce67b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:07.983613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557adbcb-332d-4d0d-9794-4b6e0fce67b9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:07.983915 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557adbcb-332d-4d0d-9794-4b6e0fce67b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:07.984185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557adbcb-332d-4d0d-9794-4b6e0fce67b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:07.994471 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557adbcb-332d-4d0d-9794-4b6e0fce67b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-557adbcb-332d-4d0d-9794-4b6e0fce67b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:48:07.999560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:08.005641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:08.006651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:08.006651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:08.006651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:08.006651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:08.006651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:48:08.006651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:48:08.006651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:48:08.006651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:48:08.006651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:48:08.006651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:48:08.006651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:48:08.006651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:08.006651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 612/1222] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:48:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:08.007483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44585ab5-7398-466e-ab8a-c10f797888c0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:08.106757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44585ab5-7398-466e-ab8a-c10f797888c0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:08.130587 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:08.142064 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b6e51-a818-42e2-a72b-3c54cc18cbed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:08.142682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b6e51-a818-42e2-a72b-3c54cc18cbed None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:08.142966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b6e51-a818-42e2-a72b-3c54cc18cbed None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:08.143231 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b6e51-a818-42e2-a72b-3c54cc18cbed None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:08.143925 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b6e51-a818-42e2-a72b-3c54cc18cbed None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:08.144220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b6e51-a818-42e2-a72b-3c54cc18cbed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:08.145642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156b6e51-a818-42e2-a72b-3c54cc18cbed None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:08.154173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 611/1223] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:48:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:08.160959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557adbcb-332d-4d0d-9794-4b6e0fce67b9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:08.209969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:08.273463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557adbcb-332d-4d0d-9794-4b6e0fce67b9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:08.307073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96d2c7-a889-4de9-bffe-d2b164188b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:08.307714 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96d2c7-a889-4de9-bffe-d2b164188b3d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:08.307997 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96d2c7-a889-4de9-bffe-d2b164188b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:08.308268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d96d2c7-a889-4de9-bffe-d2b164188b3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:08.318790 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96d2c7-a889-4de9-bffe-d2b164188b3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:08.320285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d96d2c7-a889-4de9-bffe-d2b164188b3d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0d96d2c7-a889-4de9-bffe-d2b164188b3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:48:08.335194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:08.336341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:48:08.338131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:48:08.338131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:08.338131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:08.338131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 613/1224] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:08.342934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b6e51-a818-42e2-a72b-3c54cc18cbed None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:08.445010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b6e51-a818-42e2-a72b-3c54cc18cbed None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:08.479663 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ee6b3-2673-40b0-97fe-24a924ce172e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:08.480319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ee6b3-2673-40b0-97fe-24a924ce172e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:08.480566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ee6b3-2673-40b0-97fe-24a924ce172e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:08.480837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ee6b3-2673-40b0-97fe-24a924ce172e None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:08.481475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ee6b3-2673-40b0-97fe-24a924ce172e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:08.481777 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ee6b3-2673-40b0-97fe-24a924ce172e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:08.484061 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ee6b3-2673-40b0-97fe-24a924ce172e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:08.495700 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 612/1225] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:08.502053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96d2c7-a889-4de9-bffe-d2b164188b3d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:08.600352 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:08.606988 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d96d2c7-a889-4de9-bffe-d2b164188b3d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:08.653741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb0257-0ebb-48d7-a437-5ed6deaff70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:08.654359 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb0257-0ebb-48d7-a437-5ed6deaff70c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:08.655186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb0257-0ebb-48d7-a437-5ed6deaff70c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:08.655551 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bb0257-0ebb-48d7-a437-5ed6deaff70c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:08.663379 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb0257-0ebb-48d7-a437-5ed6deaff70c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:08.664853 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bb0257-0ebb-48d7-a437-5ed6deaff70c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e5bb0257-0ebb-48d7-a437-5ed6deaff70c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:48:08.676596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:08.677685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:08.678711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:08.678711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:08.678711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:48:08.678711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:48:08.678711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:48:08.678711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:48:08.678711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:48:08.678711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:48:08.678711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:48:08.678711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:08.678711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 614/1226] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:08.683803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ee6b3-2673-40b0-97fe-24a924ce172e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:08.730795 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:08.783834 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ee6b3-2673-40b0-97fe-24a924ce172e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:08.817944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d930007-e7d9-4afb-b76e-cee51ffcbe4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:08.818629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d930007-e7d9-4afb-b76e-cee51ffcbe4a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:08.818911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d930007-e7d9-4afb-b76e-cee51ffcbe4a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:08.819184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d930007-e7d9-4afb-b76e-cee51ffcbe4a None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:08.819929 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d930007-e7d9-4afb-b76e-cee51ffcbe4a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:08.820232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d930007-e7d9-4afb-b76e-cee51ffcbe4a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:08.821707 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d930007-e7d9-4afb-b76e-cee51ffcbe4a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:08.840266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 613/1227] 10.222.0.67 () {56 vars in 1143 bytes} [Mon Sep 26 17:48:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:08.847032 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb0257-0ebb-48d7-a437-5ed6deaff70c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:08.948717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bb0257-0ebb-48d7-a437-5ed6deaff70c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:08.987286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e2a94-0174-4eea-b8ce-27bbffa01923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:08.988131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e2a94-0174-4eea-b8ce-27bbffa01923 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:08.988408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e2a94-0174-4eea-b8ce-27bbffa01923 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:08.988675 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e2a94-0174-4eea-b8ce-27bbffa01923 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:08.999636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e2a94-0174-4eea-b8ce-27bbffa01923 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:09.001093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e2a94-0174-4eea-b8ce-27bbffa01923 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-881e2a94-0174-4eea-b8ce-27bbffa01923 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:48:09.016020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:09.017172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:48:09.018412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:48:09.018412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:09.018412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:09.018412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 615/1228] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:09.022973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d930007-e7d9-4afb-b76e-cee51ffcbe4a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:09.122681 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d930007-e7d9-4afb-b76e-cee51ffcbe4a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:09.125576 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:09.157727 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2835dc-37ef-45a9-aa37-cae17c5e0cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:09.158362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2835dc-37ef-45a9-aa37-cae17c5e0cfd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:09.158659 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2835dc-37ef-45a9-aa37-cae17c5e0cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:09.159039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2835dc-37ef-45a9-aa37-cae17c5e0cfd None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:09.159695 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2835dc-37ef-45a9-aa37-cae17c5e0cfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:09.159999 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2835dc-37ef-45a9-aa37-cae17c5e0cfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:09.161359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2835dc-37ef-45a9-aa37-cae17c5e0cfd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:09.172328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 614/1229] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:48:09.178190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e2a94-0174-4eea-b8ce-27bbffa01923 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:09.188211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:09.278010 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e2a94-0174-4eea-b8ce-27bbffa01923 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:09.311444 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f38f02-a968-482a-819e-456404d3ce1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:09.312266 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f38f02-a968-482a-819e-456404d3ce1d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:09.312546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f38f02-a968-482a-819e-456404d3ce1d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:09.312817 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f38f02-a968-482a-819e-456404d3ce1d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:09.319219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f38f02-a968-482a-819e-456404d3ce1d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:09.320681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f38f02-a968-482a-819e-456404d3ce1d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:09.332401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-19f38f02-a968-482a-819e-456404d3ce1d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:48:09.332401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:09.332401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:09.332401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:09.332401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:09.332401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:09.332401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:09.332401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:09.332401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:09.332401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:09.332401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:09.332401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:09.332401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:48:09.332401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:48:09.332401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:09.339036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:09.340120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:48:09.340120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:48:09.340120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:48:09.340120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:48:09.340120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:48:09.340120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:48:09.340120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:48:09.340120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:09.340120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 616/1230] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:48:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:09.340590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2835dc-37ef-45a9-aa37-cae17c5e0cfd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:09.439261 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2835dc-37ef-45a9-aa37-cae17c5e0cfd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:09.473339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31744606-1c31-4c8e-9c50-5cbd289ed5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:09.473942 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31744606-1c31-4c8e-9c50-5cbd289ed5e0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:09.474222 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31744606-1c31-4c8e-9c50-5cbd289ed5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:09.474491 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31744606-1c31-4c8e-9c50-5cbd289ed5e0 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:09.475167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31744606-1c31-4c8e-9c50-5cbd289ed5e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:09.475468 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31744606-1c31-4c8e-9c50-5cbd289ed5e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:09.477034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31744606-1c31-4c8e-9c50-5cbd289ed5e0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:09.488772 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 615/1231] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:48:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:09.494837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f38f02-a968-482a-819e-456404d3ce1d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:09.560555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:09.595472 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f38f02-a968-482a-819e-456404d3ce1d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:09.627929 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f659255-8314-44f0-a13e-77e8f112d5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:09.628576 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f659255-8314-44f0-a13e-77e8f112d5a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:09.628854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f659255-8314-44f0-a13e-77e8f112d5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:09.629119 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f659255-8314-44f0-a13e-77e8f112d5a0 None admin] PATH_INFO: `/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/22a512e0a64e420abf15b2150ccc2b32/roles/90322db354f94f03b32ec088922baf2e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:09.643842 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f659255-8314-44f0-a13e-77e8f112d5a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=71c96e5caeb44927bc21504fe54f0f98, user_id=22a512e0a64e420abf15b2150ccc2b32, role_id=90322db354f94f03b32ec088922baf2e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:09.645330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f659255-8314-44f0-a13e-77e8f112d5a0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:09.736609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 617/1232] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:48:09 2022] PUT /identity/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/22a512e0a64e420abf15b2150ccc2b32/roles/90322db354f94f03b32ec088922baf2e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:48:11.901240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ae183-928d-4ced-8e74-5f7b26f2b832 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:11.901770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ae183-928d-4ced-8e74-5f7b26f2b832 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:11.901770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8ae183-928d-4ced-8e74-5f7b26f2b832 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:11.903108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 616/1233] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:48:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:48:11.908988 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abc1ec-c309-483e-8d7f-925a855b61f2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:11.909385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abc1ec-c309-483e-8d7f-925a855b61f2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:11.909540 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8abc1ec-c309-483e-8d7f-925a855b61f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:11.927541 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f8abc1ec-c309-483e-8d7f-925a855b61f2 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:48:11.928778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f8abc1ec-c309-483e-8d7f-925a855b61f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:48:11.995508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 618/1234] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:48:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:48:12.267196 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ae183-928d-4ced-8e74-5f7b26f2b832 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:12.292352 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:12.368687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8ae183-928d-4ced-8e74-5f7b26f2b832 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:12.402847 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d4711-eff7-4205-9f54-7a2588537b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:12.403478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d4711-eff7-4205-9f54-7a2588537b46 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:12.403797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d4711-eff7-4205-9f54-7a2588537b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:12.404071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d4711-eff7-4205-9f54-7a2588537b46 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:12.414734 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71d4711-eff7-4205-9f54-7a2588537b46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d71d4711-eff7-4205-9f54-7a2588537b46 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:48:12.419716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:12.424327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:12.425369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:12.425369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:12.425369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:12.425369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:12.425369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:48:12.425369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:48:12.425369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:48:12.425369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:48:12.425369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:48:12.425369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:48:12.425369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 26 17:48:12.425369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:12.425369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 617/1235] 10.222.0.67 () {56 vars in 1102 bytes} [Mon Sep 26 17:48:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:12.428830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abc1ec-c309-483e-8d7f-925a855b61f2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:12.477755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:12.537797 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8abc1ec-c309-483e-8d7f-925a855b61f2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:12.573845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851096fc-aa2e-4d87-8e8a-793e158db113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:12.574499 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851096fc-aa2e-4d87-8e8a-793e158db113 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:12.574776 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851096fc-aa2e-4d87-8e8a-793e158db113 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:12.575045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851096fc-aa2e-4d87-8e8a-793e158db113 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:12.575742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851096fc-aa2e-4d87-8e8a-793e158db113 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:12.576042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851096fc-aa2e-4d87-8e8a-793e158db113 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:12.578185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851096fc-aa2e-4d87-8e8a-793e158db113 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:12.586742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 619/1236] 10.222.0.67 () {56 vars in 1103 bytes} [Mon Sep 26 17:48:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:12.592857 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d4711-eff7-4205-9f54-7a2588537b46 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:12.677004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:12.692302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d4711-eff7-4205-9f54-7a2588537b46 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:12.726283 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bac622-2879-4bce-aea8-25ce2c5ba4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:12.726906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bac622-2879-4bce-aea8-25ce2c5ba4ef None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:12.727188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bac622-2879-4bce-aea8-25ce2c5ba4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:12.727460 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bac622-2879-4bce-aea8-25ce2c5ba4ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:12.738037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bac622-2879-4bce-aea8-25ce2c5ba4ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:12.739469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bac622-2879-4bce-aea8-25ce2c5ba4ef None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-77bac622-2879-4bce-aea8-25ce2c5ba4ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:48:12.754325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:12.760244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:48:12.761393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:48:12.761393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:12.761393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:12.761393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 618/1237] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:12.761673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851096fc-aa2e-4d87-8e8a-793e158db113 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:12.862783 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851096fc-aa2e-4d87-8e8a-793e158db113 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:12.871416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:12.897513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab8a1f-6d79-46b7-abe6-8bc2440e728a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:12.898114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab8a1f-6d79-46b7-abe6-8bc2440e728a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:12.898387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab8a1f-6d79-46b7-abe6-8bc2440e728a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:12.898655 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ab8a1f-6d79-46b7-abe6-8bc2440e728a None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:12.899279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab8a1f-6d79-46b7-abe6-8bc2440e728a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:12.899581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab8a1f-6d79-46b7-abe6-8bc2440e728a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:12.901124 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ab8a1f-6d79-46b7-abe6-8bc2440e728a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:12.912616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 620/1238] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:12.918628 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bac622-2879-4bce-aea8-25ce2c5ba4ef None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:13.030743 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bac622-2879-4bce-aea8-25ce2c5ba4ef None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:13.065086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb67314-65b2-4e52-b448-b5e70448d178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:13.065715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb67314-65b2-4e52-b448-b5e70448d178 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:13.065987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb67314-65b2-4e52-b448-b5e70448d178 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:13.066253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb67314-65b2-4e52-b448-b5e70448d178 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:13.068983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:13.073961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb67314-65b2-4e52-b448-b5e70448d178 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:13.075376 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb67314-65b2-4e52-b448-b5e70448d178 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5bb67314-65b2-4e52-b448-b5e70448d178 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:48:13.087278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:13.088480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:13.089634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:13.089634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:13.089634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:48:13.089634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:48:13.089634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:48:13.089634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:48:13.089634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:48:13.089634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:48:13.089634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:48:13.089634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:13.089634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 619/1239] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:13.094550 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab8a1f-6d79-46b7-abe6-8bc2440e728a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:13.193960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ab8a1f-6d79-46b7-abe6-8bc2440e728a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:13.229421 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13fe6d-8fed-466b-aec3-8a573416022b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:13.230032 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13fe6d-8fed-466b-aec3-8a573416022b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:13.230310 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13fe6d-8fed-466b-aec3-8a573416022b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:13.230583 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13fe6d-8fed-466b-aec3-8a573416022b None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:13.231303 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab13fe6d-8fed-466b-aec3-8a573416022b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:13.231605 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab13fe6d-8fed-466b-aec3-8a573416022b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:13.234277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab13fe6d-8fed-466b-aec3-8a573416022b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:13.253466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 621/1240] 10.222.0.67 () {56 vars in 1143 bytes} [Mon Sep 26 17:48:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:13.259394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb67314-65b2-4e52-b448-b5e70448d178 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:13.364409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb67314-65b2-4e52-b448-b5e70448d178 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:13.400020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2f6f1-f9bb-456d-bbf8-6b01def980e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:13.400629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2f6f1-f9bb-456d-bbf8-6b01def980e2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:13.400906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2f6f1-f9bb-456d-bbf8-6b01def980e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:13.401174 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2f6f1-f9bb-456d-bbf8-6b01def980e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:13.462561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:13.469614 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2f6f1-f9bb-456d-bbf8-6b01def980e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:13.471053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f2f6f1-f9bb-456d-bbf8-6b01def980e2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:13.570844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 620/1241] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:48:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:48:13.576739 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13fe6d-8fed-466b-aec3-8a573416022b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:13.611268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:13.682608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13fe6d-8fed-466b-aec3-8a573416022b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:13.715524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e427d22-8b24-47f4-bddd-3d059eb70b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:13.716114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e427d22-8b24-47f4-bddd-3d059eb70b37 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:13.716392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e427d22-8b24-47f4-bddd-3d059eb70b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:13.716658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e427d22-8b24-47f4-bddd-3d059eb70b37 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:13.727127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e427d22-8b24-47f4-bddd-3d059eb70b37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:13.728576 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e427d22-8b24-47f4-bddd-3d059eb70b37 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-9e427d22-8b24-47f4-bddd-3d059eb70b37 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:48:13.743574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:13.744741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:48:13.745866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:48:13.745866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:13.745866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:13.745866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 622/1242] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:13.750590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2f6f1-f9bb-456d-bbf8-6b01def980e2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:13.845048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:13.850849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2f6f1-f9bb-456d-bbf8-6b01def980e2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:13.884868 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15089899-4ed1-4b0a-adb2-be9b3f11dba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:13.885467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15089899-4ed1-4b0a-adb2-be9b3f11dba9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:13.885748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15089899-4ed1-4b0a-adb2-be9b3f11dba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:13.886021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15089899-4ed1-4b0a-adb2-be9b3f11dba9 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:13.886692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15089899-4ed1-4b0a-adb2-be9b3f11dba9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:13.887283 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15089899-4ed1-4b0a-adb2-be9b3f11dba9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:13.888675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15089899-4ed1-4b0a-adb2-be9b3f11dba9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:13.899912 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 621/1243] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:13.906505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e427d22-8b24-47f4-bddd-3d059eb70b37 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:13.986962 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:14.007923 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e427d22-8b24-47f4-bddd-3d059eb70b37 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:14.041662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc633ad6-fa83-483f-9a7c-603ffd347407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:14.042286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc633ad6-fa83-483f-9a7c-603ffd347407 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:14.042566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc633ad6-fa83-483f-9a7c-603ffd347407 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:14.042836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc633ad6-fa83-483f-9a7c-603ffd347407 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:14.049637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc633ad6-fa83-483f-9a7c-603ffd347407 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:14.051059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc633ad6-fa83-483f-9a7c-603ffd347407 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:14.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-fc633ad6-fa83-483f-9a7c-603ffd347407 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:48:14.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:14.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:14.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:14.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:14.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:14.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:14.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:14.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:14.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:14.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:14.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:14.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:48:14.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:48:14.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:14.063209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:14.064337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:48:14.064337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:48:14.064337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:48:14.064337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:48:14.064337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:48:14.064337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:48:14.064337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:48:14.064337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:14.064337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 623/1244] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:48:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:14.070997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15089899-4ed1-4b0a-adb2-be9b3f11dba9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:14.172778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15089899-4ed1-4b0a-adb2-be9b3f11dba9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:14.206203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21358655-d126-45dc-b026-2dafc88b58b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:14.206808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21358655-d126-45dc-b026-2dafc88b58b4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:14.207090 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21358655-d126-45dc-b026-2dafc88b58b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:14.207366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21358655-d126-45dc-b026-2dafc88b58b4 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:14.208081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21358655-d126-45dc-b026-2dafc88b58b4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:14.208389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21358655-d126-45dc-b026-2dafc88b58b4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:14.209828 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21358655-d126-45dc-b026-2dafc88b58b4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:14.222841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 622/1245] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:48:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:14.229106 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc633ad6-fa83-483f-9a7c-603ffd347407 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:14.329158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc633ad6-fa83-483f-9a7c-603ffd347407 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:14.362824 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbfe33-47e8-413e-9128-856d11b137bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:14.363437 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbfe33-47e8-413e-9128-856d11b137bb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:14.363760 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbfe33-47e8-413e-9128-856d11b137bb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:14.364141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbfe33-47e8-413e-9128-856d11b137bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:14.364766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbfe33-47e8-413e-9128-856d11b137bb None admin] RBAC: Adding query filter params (role.id=90322db354f94f03b32ec088922baf2e, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=22a512e0a64e420abf15b2150ccc2b32) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:14.365079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbfe33-47e8-413e-9128-856d11b137bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:14.366525 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfbfe33-47e8-413e-9128-856d11b137bb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:14.372615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 624/1246] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:48:14 2022] GET /identity/v3/role_assignments?user.id=22a512e0a64e420abf15b2150ccc2b32&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=90322db354f94f03b32ec088922baf2e => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:16.577685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66258dd4-f7ed-472b-bc4b-b50cf89f2501 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:16.578265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66258dd4-f7ed-472b-bc4b-b50cf89f2501 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:16.578265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66258dd4-f7ed-472b-bc4b-b50cf89f2501 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:16.579547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 623/1247] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:48:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:48:16.584697 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d16b7d-69d1-4b8c-acc4-b73d8ba25203 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:16.585092 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d16b7d-69d1-4b8c-acc4-b73d8ba25203 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:16.585240 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d16b7d-69d1-4b8c-acc4-b73d8ba25203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:16.591570 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:16.604201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-05d16b7d-69d1-4b8c-acc4-b73d8ba25203 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:48:16.605387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-05d16b7d-69d1-4b8c-acc4-b73d8ba25203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:48:16.678284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 625/1248] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:48:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:48:16.949765 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66258dd4-f7ed-472b-bc4b-b50cf89f2501 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:16.968581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:17.055188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66258dd4-f7ed-472b-bc4b-b50cf89f2501 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:17.089206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad037ce-fdaf-42be-a324-f52d4d75ae1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:17.089924 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad037ce-fdaf-42be-a324-f52d4d75ae1c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:17.090233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad037ce-fdaf-42be-a324-f52d4d75ae1c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:17.090509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad037ce-fdaf-42be-a324-f52d4d75ae1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:17.100861 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad037ce-fdaf-42be-a324-f52d4d75ae1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4ad037ce-fdaf-42be-a324-f52d4d75ae1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:48:17.105708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:17.106804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:17.108207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:17.108207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:17.108207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:17.108207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:17.108207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:48:17.108207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:48:17.108207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:48:17.108207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:48:17.108207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:48:17.108207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:48:17.108207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:48:17.108207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:17.108207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 624/1249] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:48:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:17.113688 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d16b7d-69d1-4b8c-acc4-b73d8ba25203 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:17.210143 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d16b7d-69d1-4b8c-acc4-b73d8ba25203 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:17.238992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:17.243989 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e500ac-d696-4839-83f9-65bf089e5236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:17.244640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e500ac-d696-4839-83f9-65bf089e5236 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:17.244922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e500ac-d696-4839-83f9-65bf089e5236 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:17.245196 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e500ac-d696-4839-83f9-65bf089e5236 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:17.245856 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e500ac-d696-4839-83f9-65bf089e5236 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:17.246157 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e500ac-d696-4839-83f9-65bf089e5236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:17.247523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e500ac-d696-4839-83f9-65bf089e5236 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:17.256251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 626/1250] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:48:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:17.262382 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad037ce-fdaf-42be-a324-f52d4d75ae1c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:17.338947 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:17.362510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad037ce-fdaf-42be-a324-f52d4d75ae1c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:17.396206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfe84e-09d5-4dcc-8dd1-c8b7e8068e1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:17.396834 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfe84e-09d5-4dcc-8dd1-c8b7e8068e1a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:17.397110 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfe84e-09d5-4dcc-8dd1-c8b7e8068e1a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:17.397529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cfe84e-09d5-4dcc-8dd1-c8b7e8068e1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:17.407848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cfe84e-09d5-4dcc-8dd1-c8b7e8068e1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:17.409277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cfe84e-09d5-4dcc-8dd1-c8b7e8068e1a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-08cfe84e-09d5-4dcc-8dd1-c8b7e8068e1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:48:17.423956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:17.425053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:48:17.425971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:48:17.425971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:17.425971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:17.425971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 625/1251] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:17.431570 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e500ac-d696-4839-83f9-65bf089e5236 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:17.531618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e500ac-d696-4839-83f9-65bf089e5236 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:17.566535 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99ff59-2991-4569-a247-be5104db24e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:17.567150 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99ff59-2991-4569-a247-be5104db24e5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:17.567429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99ff59-2991-4569-a247-be5104db24e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:17.567736 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99ff59-2991-4569-a247-be5104db24e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:17.568381 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99ff59-2991-4569-a247-be5104db24e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:17.568687 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99ff59-2991-4569-a247-be5104db24e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:17.570047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99ff59-2991-4569-a247-be5104db24e5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:17.583702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 627/1252] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:17.589156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfe84e-09d5-4dcc-8dd1-c8b7e8068e1a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:17.699766 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cfe84e-09d5-4dcc-8dd1-c8b7e8068e1a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:17.726682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:17.735944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc21e3d-2386-4d4d-9b60-ded76e8e3440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:17.736585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc21e3d-2386-4d4d-9b60-ded76e8e3440 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:17.736869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc21e3d-2386-4d4d-9b60-ded76e8e3440 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:17.737140 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc21e3d-2386-4d4d-9b60-ded76e8e3440 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:17.744723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc21e3d-2386-4d4d-9b60-ded76e8e3440 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:17.746185 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc21e3d-2386-4d4d-9b60-ded76e8e3440 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0dc21e3d-2386-4d4d-9b60-ded76e8e3440 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:48:17.758568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:17.759556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:17.760594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:17.760594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:17.760594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:48:17.760594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:48:17.760594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:48:17.760594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:48:17.760594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:48:17.760594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:48:17.760594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:48:17.760594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:17.760594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 626/1253] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:17.765897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99ff59-2991-4569-a247-be5104db24e5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:17.827274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:17.870915 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99ff59-2991-4569-a247-be5104db24e5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:17.906462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62364949-a495-40d4-9805-264ede66f36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:17.907075 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62364949-a495-40d4-9805-264ede66f36c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:17.907356 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62364949-a495-40d4-9805-264ede66f36c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:17.907627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62364949-a495-40d4-9805-264ede66f36c None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:17.908371 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62364949-a495-40d4-9805-264ede66f36c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:17.908676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62364949-a495-40d4-9805-264ede66f36c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:17.910094 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62364949-a495-40d4-9805-264ede66f36c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:17.927876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 628/1254] 10.222.0.67 () {56 vars in 1143 bytes} [Mon Sep 26 17:48:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:17.933763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc21e3d-2386-4d4d-9b60-ded76e8e3440 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:18.041050 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc21e3d-2386-4d4d-9b60-ded76e8e3440 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:18.076571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a5b77-1c34-401d-99f5-0e4e5b2f539f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:18.077125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a5b77-1c34-401d-99f5-0e4e5b2f539f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:18.077451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a5b77-1c34-401d-99f5-0e4e5b2f539f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:18.077677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a5b77-1c34-401d-99f5-0e4e5b2f539f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:18.132500 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:18.144790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a5b77-1c34-401d-99f5-0e4e5b2f539f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:18.146238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2a5b77-1c34-401d-99f5-0e4e5b2f539f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:18.246058 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 627/1255] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:48:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:48:18.252098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62364949-a495-40d4-9805-264ede66f36c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:18.353446 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62364949-a495-40d4-9805-264ede66f36c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:18.366994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:18.387927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e62a8-7039-4496-9174-2ee4ee07a494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:18.388640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e62a8-7039-4496-9174-2ee4ee07a494 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:18.388933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e62a8-7039-4496-9174-2ee4ee07a494 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:18.389196 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712e62a8-7039-4496-9174-2ee4ee07a494 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:18.399046 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e62a8-7039-4496-9174-2ee4ee07a494 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:18.400465 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712e62a8-7039-4496-9174-2ee4ee07a494 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-712e62a8-7039-4496-9174-2ee4ee07a494 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:48:18.415261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:18.416868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:48:18.418451 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:48:18.418451 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:18.418451 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:18.418451 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 629/1256] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:18.423350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a5b77-1c34-401d-99f5-0e4e5b2f539f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:18.511322 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:18.524003 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a5b77-1c34-401d-99f5-0e4e5b2f539f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:18.558486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b8a0f-ed53-4988-b10e-4ee5069313a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:18.559099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b8a0f-ed53-4988-b10e-4ee5069313a8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:18.559375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b8a0f-ed53-4988-b10e-4ee5069313a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:18.559664 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b8a0f-ed53-4988-b10e-4ee5069313a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:18.560368 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b8a0f-ed53-4988-b10e-4ee5069313a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:18.560892 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b8a0f-ed53-4988-b10e-4ee5069313a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:18.562356 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34b8a0f-ed53-4988-b10e-4ee5069313a8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:18.573259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 628/1257] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:18.579964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e62a8-7039-4496-9174-2ee4ee07a494 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:18.681419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712e62a8-7039-4496-9174-2ee4ee07a494 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:18.714388 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf45530-3631-406d-8a2b-190ddea6d4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:18.715016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf45530-3631-406d-8a2b-190ddea6d4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:18.715298 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf45530-3631-406d-8a2b-190ddea6d4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:18.715572 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf45530-3631-406d-8a2b-190ddea6d4b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:18.722635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf45530-3631-406d-8a2b-190ddea6d4b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:18.724099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf45530-3631-406d-8a2b-190ddea6d4b1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:18.736162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1cf45530-3631-406d-8a2b-190ddea6d4b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:48:18.736162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:18.736162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:18.736162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:18.736162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:18.736162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:18.736162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:18.736162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:18.736162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:18.736162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:18.736162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:18.736162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:18.736162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:48:18.736162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:48:18.736162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:18.737211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:18.738158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:48:18.738158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:48:18.738158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:48:18.738158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:48:18.738158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:48:18.738158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:48:18.738158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:48:18.738158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:18.738158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 630/1258] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:48:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:18.743256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b8a0f-ed53-4988-b10e-4ee5069313a8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:18.849300 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b8a0f-ed53-4988-b10e-4ee5069313a8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:18.883610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6d0d1-16e8-47e9-82f5-47ac21c9bf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:18.884228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d0d1-16e8-47e9-82f5-47ac21c9bf71 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:18.884513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d0d1-16e8-47e9-82f5-47ac21c9bf71 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:18.884787 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6d0d1-16e8-47e9-82f5-47ac21c9bf71 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:18.885468 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6d0d1-16e8-47e9-82f5-47ac21c9bf71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:18.885771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6d0d1-16e8-47e9-82f5-47ac21c9bf71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:18.887502 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6d0d1-16e8-47e9-82f5-47ac21c9bf71 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:18.899975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 629/1259] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:48:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:18.907198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf45530-3631-406d-8a2b-190ddea6d4b1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:18.933485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:19.010127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf45530-3631-406d-8a2b-190ddea6d4b1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:19.046043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46467579-285b-4013-b678-4494ba575da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:19.046735 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46467579-285b-4013-b678-4494ba575da0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:19.047031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46467579-285b-4013-b678-4494ba575da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:19.047302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46467579-285b-4013-b678-4494ba575da0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:19.048058 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46467579-285b-4013-b678-4494ba575da0 None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=22a512e0a64e420abf15b2150ccc2b32) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:19.048362 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46467579-285b-4013-b678-4494ba575da0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:19.049781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46467579-285b-4013-b678-4494ba575da0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:19.059860 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 631/1260] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:48:18 2022] GET /identity/v3/role_assignments?user.id=22a512e0a64e420abf15b2150ccc2b32&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=07626899472b4e32adc0c71ce796a25d => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:21.221683 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9085-735f-40c2-ae3c-82378f164e4f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:21.222211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9085-735f-40c2-ae3c-82378f164e4f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:21.222211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9085-735f-40c2-ae3c-82378f164e4f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:21.223541 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 630/1261] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:48:21.228874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7a3f4-a54b-4ade-8d2d-8c3adebf77dd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:21.229302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7a3f4-a54b-4ade-8d2d-8c3adebf77dd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:21.229769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7a3f4-a54b-4ade-8d2d-8c3adebf77dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:21.249718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bbe7a3f4-a54b-4ade-8d2d-8c3adebf77dd None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:48:21.250892 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bbe7a3f4-a54b-4ade-8d2d-8c3adebf77dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:48:21.317760 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 632/1262] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:48:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 26 17:48:21.588304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca9085-735f-40c2-ae3c-82378f164e4f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:21.600136 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:21.691923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca9085-735f-40c2-ae3c-82378f164e4f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:21.725352 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c62a20-dd1c-4901-8e3d-f22314f62e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:21.725971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c62a20-dd1c-4901-8e3d-f22314f62e16 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:21.726251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c62a20-dd1c-4901-8e3d-f22314f62e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:21.726523 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c62a20-dd1c-4901-8e3d-f22314f62e16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:21.736936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c62a20-dd1c-4901-8e3d-f22314f62e16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-04c62a20-dd1c-4901-8e3d-f22314f62e16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:48:21.741699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:21.749182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:21.750266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:21.750266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:21.750266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:21.750266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:21.750266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:48:21.750266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:48:21.750266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:48:21.750266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:48:21.750266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:48:21.750266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:48:21.750266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:48:21.750266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:21.750266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 631/1263] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:48:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:21.751115 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7a3f4-a54b-4ade-8d2d-8c3adebf77dd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:21.767539 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:21.845944 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7a3f4-a54b-4ade-8d2d-8c3adebf77dd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:21.879542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d3d84-717c-4cd6-9f9f-9ae43226ee39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:21.880168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d3d84-717c-4cd6-9f9f-9ae43226ee39 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:21.880444 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d3d84-717c-4cd6-9f9f-9ae43226ee39 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:21.880714 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d3d84-717c-4cd6-9f9f-9ae43226ee39 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:21.881427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d3d84-717c-4cd6-9f9f-9ae43226ee39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:21.881727 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d3d84-717c-4cd6-9f9f-9ae43226ee39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:21.883077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d3d84-717c-4cd6-9f9f-9ae43226ee39 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:21.891958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 633/1264] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:48:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:21.897658 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c62a20-dd1c-4901-8e3d-f22314f62e16 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:21.964127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:21.999673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c62a20-dd1c-4901-8e3d-f22314f62e16 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:22.035147 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d52cb5-67a6-4255-9bb8-0bb969089b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:22.035794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d52cb5-67a6-4255-9bb8-0bb969089b6e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:22.036072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d52cb5-67a6-4255-9bb8-0bb969089b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:22.036441 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d52cb5-67a6-4255-9bb8-0bb969089b6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:22.046923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d52cb5-67a6-4255-9bb8-0bb969089b6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:22.048432 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d52cb5-67a6-4255-9bb8-0bb969089b6e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f7d52cb5-67a6-4255-9bb8-0bb969089b6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:48:22.063319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:22.064471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:48:22.065350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:48:22.065350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:22.065350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:22.065350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 632/1265] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:22.070446 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d3d84-717c-4cd6-9f9f-9ae43226ee39 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:22.153178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:22.173343 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d3d84-717c-4cd6-9f9f-9ae43226ee39 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:22.206291 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ec417-c3cd-4676-b2d6-d88ae97eda3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:22.206936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ec417-c3cd-4676-b2d6-d88ae97eda3b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:22.207220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ec417-c3cd-4676-b2d6-d88ae97eda3b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:22.207497 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3ec417-c3cd-4676-b2d6-d88ae97eda3b None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:22.208165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ec417-c3cd-4676-b2d6-d88ae97eda3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:22.208472 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ec417-c3cd-4676-b2d6-d88ae97eda3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:22.209887 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3ec417-c3cd-4676-b2d6-d88ae97eda3b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:22.222416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 634/1266] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:22.228031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d52cb5-67a6-4255-9bb8-0bb969089b6e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:22.327876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d52cb5-67a6-4255-9bb8-0bb969089b6e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:22.347722 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:22.362602 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775c3cd-9004-4851-a1e9-a51e8765578e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:22.363233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775c3cd-9004-4851-a1e9-a51e8765578e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:22.363516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775c3cd-9004-4851-a1e9-a51e8765578e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:22.363823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775c3cd-9004-4851-a1e9-a51e8765578e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:22.371215 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775c3cd-9004-4851-a1e9-a51e8765578e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:22.372688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775c3cd-9004-4851-a1e9-a51e8765578e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5775c3cd-9004-4851-a1e9-a51e8765578e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:48:22.384293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:22.385381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:22.386585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:22.386585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:22.386585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:48:22.386585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:48:22.386585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:48:22.386585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:48:22.386585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:48:22.386585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:48:22.386585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:48:22.386585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:22.386585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 633/1267] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:22.391640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ec417-c3cd-4676-b2d6-d88ae97eda3b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:22.491241 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3ec417-c3cd-4676-b2d6-d88ae97eda3b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:22.525027 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999530ab-4a61-4bf3-a575-9a7ea4b0575d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:22.525631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999530ab-4a61-4bf3-a575-9a7ea4b0575d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:22.525914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999530ab-4a61-4bf3-a575-9a7ea4b0575d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:22.526188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999530ab-4a61-4bf3-a575-9a7ea4b0575d None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:22.526903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999530ab-4a61-4bf3-a575-9a7ea4b0575d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:22.527210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999530ab-4a61-4bf3-a575-9a7ea4b0575d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:22.528665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999530ab-4a61-4bf3-a575-9a7ea4b0575d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:22.538506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:22.548907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 635/1268] 10.222.0.67 () {56 vars in 1143 bytes} [Mon Sep 26 17:48:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:22.554371 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775c3cd-9004-4851-a1e9-a51e8765578e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:22.655061 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775c3cd-9004-4851-a1e9-a51e8765578e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:22.689074 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0628a-d3bf-4c14-bde2-724678d06e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:22.689706 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0628a-d3bf-4c14-bde2-724678d06e8f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:22.690004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0628a-d3bf-4c14-bde2-724678d06e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:22.690261 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d0628a-d3bf-4c14-bde2-724678d06e8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:22.700744 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0628a-d3bf-4c14-bde2-724678d06e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:22.702482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d0628a-d3bf-4c14-bde2-724678d06e8f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d5d0628a-d3bf-4c14-bde2-724678d06e8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:48:22.716411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:22.718112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:48:22.719194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:48:22.719194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:22.719194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:22.719194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 634/1269] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:22.724138 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999530ab-4a61-4bf3-a575-9a7ea4b0575d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:22.825889 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999530ab-4a61-4bf3-a575-9a7ea4b0575d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:22.860602 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83572a51-b371-4f6f-82bd-c496b354f234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:22.861209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83572a51-b371-4f6f-82bd-c496b354f234 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:22.861527 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83572a51-b371-4f6f-82bd-c496b354f234 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:22.861840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83572a51-b371-4f6f-82bd-c496b354f234 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:22.862473 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83572a51-b371-4f6f-82bd-c496b354f234 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:22.862778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83572a51-b371-4f6f-82bd-c496b354f234 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:22.864175 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83572a51-b371-4f6f-82bd-c496b354f234 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:22.876067 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 636/1270] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:22.881877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0628a-d3bf-4c14-bde2-724678d06e8f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:22.913666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:22.981792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d0628a-d3bf-4c14-bde2-724678d06e8f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:23.016114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b344b-b819-4d73-a972-e9a88022332d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:23.016736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b344b-b819-4d73-a972-e9a88022332d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:23.017011 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b344b-b819-4d73-a972-e9a88022332d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:23.017279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6b344b-b819-4d73-a972-e9a88022332d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:23.023906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b344b-b819-4d73-a972-e9a88022332d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:23.025336 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6b344b-b819-4d73-a972-e9a88022332d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:23.036660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6b6b344b-b819-4d73-a972-e9a88022332d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:48:23.036660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:23.036660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:23.036660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:23.036660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:23.036660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:23.036660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:23.036660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:23.036660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:23.036660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:23.036660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:23.036660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:23.036660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:48:23.036660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:48:23.036660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:23.037952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:23.039095 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:48:23.039095 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:48:23.039095 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:48:23.039095 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:48:23.039095 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:48:23.039095 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:48:23.039095 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:48:23.039095 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:23.039095 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 635/1271] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:48:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:23.043817 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83572a51-b371-4f6f-82bd-c496b354f234 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:23.103373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:23.146692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83572a51-b371-4f6f-82bd-c496b354f234 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:23.188547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0578cb9-4047-49b2-8c88-47f53d6d3d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:23.189158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0578cb9-4047-49b2-8c88-47f53d6d3d3b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:23.189443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0578cb9-4047-49b2-8c88-47f53d6d3d3b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:23.189716 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0578cb9-4047-49b2-8c88-47f53d6d3d3b None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:23.190395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0578cb9-4047-49b2-8c88-47f53d6d3d3b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:23.190701 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0578cb9-4047-49b2-8c88-47f53d6d3d3b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:23.192185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0578cb9-4047-49b2-8c88-47f53d6d3d3b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:23.204683 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 637/1272] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:48:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:23.210944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b344b-b819-4d73-a972-e9a88022332d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:23.303532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:23.312944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6b344b-b819-4d73-a972-e9a88022332d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:23.346211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de9e1e-3a35-41de-b8e8-4e5ef533e3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:23.346861 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de9e1e-3a35-41de-b8e8-4e5ef533e3c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:23.349086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de9e1e-3a35-41de-b8e8-4e5ef533e3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:23.349086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de9e1e-3a35-41de-b8e8-4e5ef533e3c9 None admin] PATH_INFO: `/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/22a512e0a64e420abf15b2150ccc2b32/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:23.349456 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de9e1e-3a35-41de-b8e8-4e5ef533e3c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=71c96e5caeb44927bc21504fe54f0f98, user_id=22a512e0a64e420abf15b2150ccc2b32, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:23.350767 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de9e1e-3a35-41de-b8e8-4e5ef533e3c9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:23.450273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 636/1273] 10.222.0.67 () {58 vars in 1439 bytes} [Mon Sep 26 17:48:23 2022] PUT /identity/v3/projects/71c96e5caeb44927bc21504fe54f0f98/users/22a512e0a64e420abf15b2150ccc2b32/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:48:25.608231 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c58653-71c3-4d28-a564-a7451e8dcc18 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:25.608865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c58653-71c3-4d28-a564-a7451e8dcc18 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:25.608865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c58653-71c3-4d28-a564-a7451e8dcc18 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:25.610194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 638/1274] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:48:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:48:25.615332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82ba79-fc96-468f-b8b3-e0e37fbddbca None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:25.615769 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82ba79-fc96-468f-b8b3-e0e37fbddbca None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:25.615944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d82ba79-fc96-468f-b8b3-e0e37fbddbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:25.634712 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2d82ba79-fc96-468f-b8b3-e0e37fbddbca None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:48:25.635923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2d82ba79-fc96-468f-b8b3-e0e37fbddbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:48:25.701884 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 637/1275] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:48:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:48:25.986016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c58653-71c3-4d28-a564-a7451e8dcc18 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:26.087286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c58653-71c3-4d28-a564-a7451e8dcc18 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:26.106876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:26.127153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afd16e-c5fb-43f3-8c0b-fbe698e5b036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:26.129335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afd16e-c5fb-43f3-8c0b-fbe698e5b036 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:26.129335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afd16e-c5fb-43f3-8c0b-fbe698e5b036 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:26.129335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afd16e-c5fb-43f3-8c0b-fbe698e5b036 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:26.138537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afd16e-c5fb-43f3-8c0b-fbe698e5b036 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-15afd16e-c5fb-43f3-8c0b-fbe698e5b036 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 26 17:48:26.143416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:26.144595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:26.145828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:26.145828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:26.145828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:26.145828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:26.145828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 26 17:48:26.145828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 26 17:48:26.145828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 26 17:48:26.145828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 26 17:48:26.145828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 26 17:48:26.145828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 26 17:48:26.145828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 26 17:48:26.145828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:26.145828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 639/1276] 10.222.0.67 () {56 vars in 1096 bytes} [Mon Sep 26 17:48:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:26.150545 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82ba79-fc96-468f-b8b3-e0e37fbddbca None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:26.231881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:26.250461 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d82ba79-fc96-468f-b8b3-e0e37fbddbca None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:26.284101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64640c2c-5be1-45fc-ae0a-3b5ab4283499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:26.284703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64640c2c-5be1-45fc-ae0a-3b5ab4283499 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:26.284982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64640c2c-5be1-45fc-ae0a-3b5ab4283499 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:26.285254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64640c2c-5be1-45fc-ae0a-3b5ab4283499 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:26.285905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64640c2c-5be1-45fc-ae0a-3b5ab4283499 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:26.286204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64640c2c-5be1-45fc-ae0a-3b5ab4283499 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:26.287559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64640c2c-5be1-45fc-ae0a-3b5ab4283499 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:26.296306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 638/1277] 10.222.0.67 () {56 vars in 1099 bytes} [Mon Sep 26 17:48:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:26.302329 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afd16e-c5fb-43f3-8c0b-fbe698e5b036 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:26.403490 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afd16e-c5fb-43f3-8c0b-fbe698e5b036 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:26.439153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a92418-5feb-4f1c-97de-8ab3ee6463e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:26.439840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a92418-5feb-4f1c-97de-8ab3ee6463e2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:26.440114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a92418-5feb-4f1c-97de-8ab3ee6463e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:26.440382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a92418-5feb-4f1c-97de-8ab3ee6463e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:26.450188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a92418-5feb-4f1c-97de-8ab3ee6463e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:26.452257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a92418-5feb-4f1c-97de-8ab3ee6463e2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-33a92418-5feb-4f1c-97de-8ab3ee6463e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:48:26.466043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:26.467530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:48:26.468663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:48:26.468663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:26.468663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:26.468663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 640/1278] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:26.473299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64640c2c-5be1-45fc-ae0a-3b5ab4283499 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:26.576427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64640c2c-5be1-45fc-ae0a-3b5ab4283499 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:26.610065 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ffa43-eb82-4eb2-8fef-b1626373f9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:26.610668 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ffa43-eb82-4eb2-8fef-b1626373f9f1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:26.610951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ffa43-eb82-4eb2-8fef-b1626373f9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:26.611228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ffa43-eb82-4eb2-8fef-b1626373f9f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:26.611888 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ffa43-eb82-4eb2-8fef-b1626373f9f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:26.612200 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ffa43-eb82-4eb2-8fef-b1626373f9f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:26.613562 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ffa43-eb82-4eb2-8fef-b1626373f9f1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:26.625758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 639/1279] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:26.631518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a92418-5feb-4f1c-97de-8ab3ee6463e2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:26.664440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:26.733204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a92418-5feb-4f1c-97de-8ab3ee6463e2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:26.767087 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c09c8-d4b5-4910-a554-6184c6d906c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:26.767738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c09c8-d4b5-4910-a554-6184c6d906c5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:26.768015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c09c8-d4b5-4910-a554-6184c6d906c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:26.768280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c09c8-d4b5-4910-a554-6184c6d906c5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:26.775567 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c09c8-d4b5-4910-a554-6184c6d906c5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:26.777609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868c09c8-d4b5-4910-a554-6184c6d906c5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-868c09c8-d4b5-4910-a554-6184c6d906c5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 26 17:48:26.788165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:26.789364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:26.790522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:26.790522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:26.790522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 26 17:48:26.790522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 26 17:48:26.790522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 26 17:48:26.790522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 26 17:48:26.790522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 26 17:48:26.790522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 26 17:48:26.790522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 26 17:48:26.790522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:26.790522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 641/1280] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:26.795116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ffa43-eb82-4eb2-8fef-b1626373f9f1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:26.798936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:26.895332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ffa43-eb82-4eb2-8fef-b1626373f9f1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:26.928861 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5937506-ea02-492d-8099-198c282cf59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:26.929467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5937506-ea02-492d-8099-198c282cf59b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:26.929748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5937506-ea02-492d-8099-198c282cf59b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:26.930020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5937506-ea02-492d-8099-198c282cf59b None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:26.930734 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5937506-ea02-492d-8099-198c282cf59b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:26.931037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5937506-ea02-492d-8099-198c282cf59b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:26.932486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5937506-ea02-492d-8099-198c282cf59b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:26.949881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 640/1281] 10.222.0.67 () {56 vars in 1143 bytes} [Mon Sep 26 17:48:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:26.956336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c09c8-d4b5-4910-a554-6184c6d906c5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:27.049978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:27.059245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c09c8-d4b5-4910-a554-6184c6d906c5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:27.093897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf8fda-8ce7-4040-bf2a-6d7ee63c306e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:27.094508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf8fda-8ce7-4040-bf2a-6d7ee63c306e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:27.094875 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf8fda-8ce7-4040-bf2a-6d7ee63c306e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:27.095172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adf8fda-8ce7-4040-bf2a-6d7ee63c306e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:27.161559 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf8fda-8ce7-4040-bf2a-6d7ee63c306e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:27.163020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adf8fda-8ce7-4040-bf2a-6d7ee63c306e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:27.249776 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:27.259700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 642/1282] 10.222.0.67 () {58 vars in 1282 bytes} [Mon Sep 26 17:48:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 26 17:48:27.266356 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5937506-ea02-492d-8099-198c282cf59b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:27.326597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:27.363298 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5937506-ea02-492d-8099-198c282cf59b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:27.397963 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf20d98-7d1e-4838-8b74-623fae993f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:27.398584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf20d98-7d1e-4838-8b74-623fae993f49 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:27.398859 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf20d98-7d1e-4838-8b74-623fae993f49 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:27.399125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf20d98-7d1e-4838-8b74-623fae993f49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:27.409426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf20d98-7d1e-4838-8b74-623fae993f49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:27.410852 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf20d98-7d1e-4838-8b74-623fae993f49 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6bf20d98-7d1e-4838-8b74-623fae993f49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 26 17:48:27.425099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:27.426205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 26 17:48:27.427204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 26 17:48:27.427204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 26 17:48:27.427204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:27.427204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 641/1283] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:27.431957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf8fda-8ce7-4040-bf2a-6d7ee63c306e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:27.531971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adf8fda-8ce7-4040-bf2a-6d7ee63c306e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:27.565807 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbec330-5743-435c-8dfc-2fef2f2f2100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:27.567848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbec330-5743-435c-8dfc-2fef2f2f2100 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:27.567848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbec330-5743-435c-8dfc-2fef2f2f2100 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:27.567848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbec330-5743-435c-8dfc-2fef2f2f2100 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:27.567848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbec330-5743-435c-8dfc-2fef2f2f2100 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:27.568192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbec330-5743-435c-8dfc-2fef2f2f2100 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:27.569550 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbec330-5743-435c-8dfc-2fef2f2f2100 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:27.581107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 643/1284] 10.222.0.67 () {56 vars in 1109 bytes} [Mon Sep 26 17:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 17:48:27.587166 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf20d98-7d1e-4838-8b74-623fae993f49 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:27.688577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf20d98-7d1e-4838-8b74-623fae993f49 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:27.701837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:27.721598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3faa35-aead-4380-be4e-92b6f427cb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:27.722214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3faa35-aead-4380-be4e-92b6f427cb5a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:27.722490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3faa35-aead-4380-be4e-92b6f427cb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:27.722759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3faa35-aead-4380-be4e-92b6f427cb5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:27.729145 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3faa35-aead-4380-be4e-92b6f427cb5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:27.730690 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3faa35-aead-4380-be4e-92b6f427cb5a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:27.742120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5b3faa35-aead-4380-be4e-92b6f427cb5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:48:27.742120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:27.742120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:27.742120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:27.742120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:27.742120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:27.742120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:27.742120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:27.742120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:27.742120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:27.742120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:27.742120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:27.742120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 26 17:48:27.742120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 26 17:48:27.742120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:27.743141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:27.744034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 26 17:48:27.744034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 26 17:48:27.744034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 26 17:48:27.744034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 26 17:48:27.744034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 26 17:48:27.744034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 26 17:48:27.744034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 26 17:48:27.744034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 26 17:48:27.744034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 642/1285] 10.222.0.67 () {56 vars in 1111 bytes} [Mon Sep 26 17:48:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:27.749262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbec330-5743-435c-8dfc-2fef2f2f2100 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:27.799421 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:27.848849 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbec330-5743-435c-8dfc-2fef2f2f2100 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:27.882007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8cd947-b111-444c-9173-d758a1b8390d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:27.882605 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cd947-b111-444c-9173-d758a1b8390d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:27.882884 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cd947-b111-444c-9173-d758a1b8390d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:27.883194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8cd947-b111-444c-9173-d758a1b8390d None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:27.884035 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cd947-b111-444c-9173-d758a1b8390d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:27.884336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cd947-b111-444c-9173-d758a1b8390d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:27.885749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8cd947-b111-444c-9173-d758a1b8390d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:27.899114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 644/1286] 10.222.0.67 () {56 vars in 1148 bytes} [Mon Sep 26 17:48:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:27.906044 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3faa35-aead-4380-be4e-92b6f427cb5a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:28.005648 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3faa35-aead-4380-be4e-92b6f427cb5a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:28.040512 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789eda4-7d57-4e90-996b-0850f4ab01cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:28.041114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789eda4-7d57-4e90-996b-0850f4ab01cf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:28.041396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789eda4-7d57-4e90-996b-0850f4ab01cf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:28.041670 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789eda4-7d57-4e90-996b-0850f4ab01cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:28.042387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789eda4-7d57-4e90-996b-0850f4ab01cf None admin] RBAC: Adding query filter params (role.id=07626899472b4e32adc0c71ce796a25d, scope.project.id=71c96e5caeb44927bc21504fe54f0f98, user.id=22a512e0a64e420abf15b2150ccc2b32) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:28.042697 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789eda4-7d57-4e90-996b-0850f4ab01cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:28.044169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789eda4-7d57-4e90-996b-0850f4ab01cf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:28.050570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 643/1287] 10.222.0.67 () {56 vars in 1374 bytes} [Mon Sep 26 17:48:27 2022] GET /identity/v3/role_assignments?user.id=22a512e0a64e420abf15b2150ccc2b32&scope.project.id=71c96e5caeb44927bc21504fe54f0f98&role.id=07626899472b4e32adc0c71ce796a25d => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:30.280456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8290a-c883-4b1e-8fd2-70103074070a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:30.280960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8290a-c883-4b1e-8fd2-70103074070a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:30.280960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c8290a-c883-4b1e-8fd2-70103074070a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:30.282234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 645/1288] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:48:30.287397 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf8988-40d2-44f8-81a9-6c5318c78c80 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:30.287813 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf8988-40d2-44f8-81a9-6c5318c78c80 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:30.287978 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf8988-40d2-44f8-81a9-6c5318c78c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:30.306473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8daf8988-40d2-44f8-81a9-6c5318c78c80 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:48:30.307687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8daf8988-40d2-44f8-81a9-6c5318c78c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:48:30.337835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:30.376248 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 644/1289] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:48:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:48:30.645959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8290a-c883-4b1e-8fd2-70103074070a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:30.747131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c8290a-c883-4b1e-8fd2-70103074070a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:30.757528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:30.781573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906554a-d9af-4b10-9261-371a88978c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:30.782212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906554a-d9af-4b10-9261-371a88978c4e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:30.782655 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906554a-d9af-4b10-9261-371a88978c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:30.783227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f906554a-d9af-4b10-9261-371a88978c4e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:30.783840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906554a-d9af-4b10-9261-371a88978c4e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:30.785203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f906554a-d9af-4b10-9261-371a88978c4e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f906554a-d9af-4b10-9261-371a88978c4e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:30.794090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:30.796153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:30.797066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:30.797066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:30.797066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:30.797066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:30.797066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:30.797066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:30.797066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:30.797066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:30.797066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:30.797066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:30.797066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:48:30.797066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:48:30.797066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:48:30.797066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:30.797066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 646/1290] 10.222.0.67 () {56 vars in 1117 bytes} [Mon Sep 26 17:48:30 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:30.801789 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf8988-40d2-44f8-81a9-6c5318c78c80 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:30.899790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf8988-40d2-44f8-81a9-6c5318c78c80 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:30.934385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16adab06-7e3e-44d4-80ba-2078199fed35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:30.935079 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adab06-7e3e-44d4-80ba-2078199fed35 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:30.935349 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adab06-7e3e-44d4-80ba-2078199fed35 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:30.935618 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16adab06-7e3e-44d4-80ba-2078199fed35 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:30.936258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adab06-7e3e-44d4-80ba-2078199fed35 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:30.936556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adab06-7e3e-44d4-80ba-2078199fed35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:30.937917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16adab06-7e3e-44d4-80ba-2078199fed35 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:30.942594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 645/1291] 10.222.0.67 () {56 vars in 1116 bytes} [Mon Sep 26 17:48:30 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:30.948580 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906554a-d9af-4b10-9261-371a88978c4e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:31.050350 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f906554a-d9af-4b10-9261-371a88978c4e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:31.083937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0da83-7fa5-4f10-b604-ce7ac741c253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:31.084540 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0da83-7fa5-4f10-b604-ce7ac741c253 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:31.084819 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0da83-7fa5-4f10-b604-ce7ac741c253 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:31.085091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0da83-7fa5-4f10-b604-ce7ac741c253 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:31.085705 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0da83-7fa5-4f10-b604-ce7ac741c253 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:31.086004 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0da83-7fa5-4f10-b604-ce7ac741c253 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:31.087365 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0da83-7fa5-4f10-b604-ce7ac741c253 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:31.091981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 647/1292] 10.222.0.67 () {56 vars in 1116 bytes} [Mon Sep 26 17:48:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:33.273508 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2e3e8-6499-40d0-9218-93fda001577f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:33.274089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2e3e8-6499-40d0-9218-93fda001577f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:33.274089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc2e3e8-6499-40d0-9218-93fda001577f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:33.275275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 646/1293] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:48:33.281118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae40af-a79f-4fba-bba1-313b8815b062 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:33.281553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae40af-a79f-4fba-bba1-313b8815b062 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:33.281676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ae40af-a79f-4fba-bba1-313b8815b062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:33.302763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-88ae40af-a79f-4fba-bba1-313b8815b062 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:48:33.303991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-88ae40af-a79f-4fba-bba1-313b8815b062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:48:33.333452 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:33.369632 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 648/1294] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:48:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:48:33.666462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2e3e8-6499-40d0-9218-93fda001577f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:33.697606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:33.768098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc2e3e8-6499-40d0-9218-93fda001577f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:33.802090 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fdcac-63bd-4049-bf2d-f62a4bf52cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:33.802683 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fdcac-63bd-4049-bf2d-f62a4bf52cb5 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:33.802957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fdcac-63bd-4049-bf2d-f62a4bf52cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:33.803223 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fdcac-63bd-4049-bf2d-f62a4bf52cb5 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:33.803892 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05fdcac-63bd-4049-bf2d-f62a4bf52cb5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:33.805251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05fdcac-63bd-4049-bf2d-f62a4bf52cb5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:33.908343 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 647/1295] 10.222.0.67 () {60 vars in 1141 bytes} [Mon Sep 26 17:48:33 2022] POST /identity/v3/services => generated 246 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:48:36.108751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa05794-bc0c-4710-a3a9-ca98157dab5a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:36.109297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa05794-bc0c-4710-a3a9-ca98157dab5a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:36.109297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa05794-bc0c-4710-a3a9-ca98157dab5a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:36.110618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 649/1296] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:48:36.115848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62e11-3345-4347-803f-4cd050e518f7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:36.116285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62e11-3345-4347-803f-4cd050e518f7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:36.116397 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e62e11-3345-4347-803f-4cd050e518f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:36.134651 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b2e62e11-3345-4347-803f-4cd050e518f7 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:48:36.138758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b2e62e11-3345-4347-803f-4cd050e518f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:48:36.209337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 648/1297] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:48:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:48:36.485627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa05794-bc0c-4710-a3a9-ca98157dab5a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:36.587220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa05794-bc0c-4710-a3a9-ca98157dab5a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:36.621249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89a25b-0908-4acb-9c2f-29471858b68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:36.621875 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89a25b-0908-4acb-9c2f-29471858b68b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:36.622267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89a25b-0908-4acb-9c2f-29471858b68b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:36.622541 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad89a25b-0908-4acb-9c2f-29471858b68b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:36.623113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89a25b-0908-4acb-9c2f-29471858b68b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:36.624506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad89a25b-0908-4acb-9c2f-29471858b68b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ad89a25b-0908-4acb-9c2f-29471858b68b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:36.633114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:36.634267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:36.635294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:36.635294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:36.635294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:36.635294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:36.635294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:36.635294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:36.635294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:36.635294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:36.635294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:36.635294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:36.635294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:48:36.635294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:48:36.635294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:48:36.635294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:36.635294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 650/1298] 10.222.0.67 () {56 vars in 1117 bytes} [Mon Sep 26 17:48:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:36.640909 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e62e11-3345-4347-803f-4cd050e518f7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:36.660582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:36.741207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e62e11-3345-4347-803f-4cd050e518f7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:36.774993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec84a7b-0639-44bb-98ba-d51dbc475c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:36.775594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec84a7b-0639-44bb-98ba-d51dbc475c50 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:36.775903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec84a7b-0639-44bb-98ba-d51dbc475c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:36.780597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec84a7b-0639-44bb-98ba-d51dbc475c50 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:36.780597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec84a7b-0639-44bb-98ba-d51dbc475c50 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:36.780597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec84a7b-0639-44bb-98ba-d51dbc475c50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:36.780597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec84a7b-0639-44bb-98ba-d51dbc475c50 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:36.784391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 649/1299] 10.222.0.67 () {56 vars in 1116 bytes} [Mon Sep 26 17:48:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:36.790433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89a25b-0908-4acb-9c2f-29471858b68b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:36.821847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:36.889262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad89a25b-0908-4acb-9c2f-29471858b68b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:36.921632 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a10e39-b720-4169-8154-0d7f767312a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:36.922229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a10e39-b720-4169-8154-0d7f767312a0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:36.922505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a10e39-b720-4169-8154-0d7f767312a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:36.922775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a10e39-b720-4169-8154-0d7f767312a0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:36.923457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a10e39-b720-4169-8154-0d7f767312a0 None admin] RBAC: Adding query filter params (interface=public, service_id=522ef91d3c834314a3fd9295bf0b0d32, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:36.923786 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a10e39-b720-4169-8154-0d7f767312a0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:36.925133 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a10e39-b720-4169-8154-0d7f767312a0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:36.929946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 651/1300] 10.222.0.67 () {56 vars in 1251 bytes} [Mon Sep 26 17:48:36 2022] GET /identity/v3/endpoints?service_id=522ef91d3c834314a3fd9295bf0b0d32&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:36.935932 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec84a7b-0639-44bb-98ba-d51dbc475c50 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:37.020531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:37.037842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec84a7b-0639-44bb-98ba-d51dbc475c50 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:37.070838 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63429367-da05-46e4-86b0-f07b716789cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:37.071446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63429367-da05-46e4-86b0-f07b716789cd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:37.071763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63429367-da05-46e4-86b0-f07b716789cd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:37.072036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63429367-da05-46e4-86b0-f07b716789cd None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:37.072628 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63429367-da05-46e4-86b0-f07b716789cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:37.073985 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63429367-da05-46e4-86b0-f07b716789cd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:37.080935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 650/1301] 10.222.0.67 () {56 vars in 1087 bytes} [Mon Sep 26 17:48:36 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:48:39.247033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4005f-30ff-4932-8553-16812cc69c3e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:39.247537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4005f-30ff-4932-8553-16812cc69c3e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:39.247537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4005f-30ff-4932-8553-16812cc69c3e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:39.248922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 652/1302] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:48:39.254041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a2324-fe23-49b9-8e1c-2e1be664e16f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:39.264239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a2324-fe23-49b9-8e1c-2e1be664e16f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:39.264239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a2324-fe23-49b9-8e1c-2e1be664e16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:39.273539 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0d1a2324-fe23-49b9-8e1c-2e1be664e16f None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:48:39.274745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0d1a2324-fe23-49b9-8e1c-2e1be664e16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:48:39.343486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 651/1303] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:48:39 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 26 17:48:39.615399 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4005f-30ff-4932-8553-16812cc69c3e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:39.715953 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:39.722859 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4005f-30ff-4932-8553-16812cc69c3e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:39.756120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfa73f-ed16-47cb-98a6-3fd2801facb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:39.756748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfa73f-ed16-47cb-98a6-3fd2801facb6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:39.757027 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfa73f-ed16-47cb-98a6-3fd2801facb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:39.764458 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dfa73f-ed16-47cb-98a6-3fd2801facb6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:39.764458 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfa73f-ed16-47cb-98a6-3fd2801facb6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:39.764458 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dfa73f-ed16-47cb-98a6-3fd2801facb6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-26dfa73f-ed16-47cb-98a6-3fd2801facb6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 26 17:48:39.769319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 26 17:48:39.770374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 26 17:48:39.771200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 26 17:48:39.771200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 26 17:48:39.771200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 26 17:48:39.771200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 26 17:48:39.771200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 26 17:48:39.771200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 26 17:48:39.771200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 26 17:48:39.771200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 26 17:48:39.771200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 26 17:48:39.771200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 26 17:48:39.771200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 26 17:48:39.771200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 26 17:48:39.771200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 26 17:48:39.771200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 26 17:48:39.771200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 653/1304] 10.222.0.67 () {56 vars in 1117 bytes} [Mon Sep 26 17:48:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 26 17:48:39.776944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a2324-fe23-49b9-8e1c-2e1be664e16f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:39.852639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:39.876236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a2324-fe23-49b9-8e1c-2e1be664e16f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:39.911803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2e0d0a-3da7-4f8c-80f2-2dc48f5f9800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:39.912405 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e0d0a-3da7-4f8c-80f2-2dc48f5f9800 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:39.912687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e0d0a-3da7-4f8c-80f2-2dc48f5f9800 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:39.912960 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2e0d0a-3da7-4f8c-80f2-2dc48f5f9800 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:39.913579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e0d0a-3da7-4f8c-80f2-2dc48f5f9800 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:48:39.913881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e0d0a-3da7-4f8c-80f2-2dc48f5f9800 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:39.915262 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2e0d0a-3da7-4f8c-80f2-2dc48f5f9800 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:39.920665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 652/1305] 10.222.0.67 () {56 vars in 1116 bytes} [Mon Sep 26 17:48:39 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:48:39.926577 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfa73f-ed16-47cb-98a6-3fd2801facb6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:48:40.030040 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dfa73f-ed16-47cb-98a6-3fd2801facb6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:48:40.063905 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df02dee-966d-4901-81f2-4cbed6cb73c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:48:40.064510 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df02dee-966d-4901-81f2-4cbed6cb73c5 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:48:40.064794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df02dee-966d-4901-81f2-4cbed6cb73c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:48:40.065065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df02dee-966d-4901-81f2-4cbed6cb73c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:48:40.065733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df02dee-966d-4901-81f2-4cbed6cb73c5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:48:40.067134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df02dee-966d-4901-81f2-4cbed6cb73c5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:48:40.141478 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 654/1306] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 17:48:39 2022] POST /identity/v3/endpoints => generated 331 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:48:49.917300 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=30 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:48:50.136744 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:35.340720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd65d4-2967-419b-8b61-a9fdd10d4a3d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:35.341272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd65d4-2967-419b-8b61-a9fdd10d4a3d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:35.341494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd65d4-2967-419b-8b61-a9fdd10d4a3d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:35.349181 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471e356-24c3-44c1-8a78-d000c013cd13 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:35.350014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471e356-24c3-44c1-8a78-d000c013cd13 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:35.350437 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0471e356-24c3-44c1-8a78-d000c013cd13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:35.356005 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 653/1307] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:54:35 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:54:35.454032 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0471e356-24c3-44c1-8a78-d000c013cd13 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:54:35.455565 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0471e356-24c3-44c1-8a78-d000c013cd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:35.575766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 655/1308] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:54:35.865546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3373f-d6a7-4a2e-bea8-3861a29cf8b5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:35.865827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3373f-d6a7-4a2e-bea8-3861a29cf8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:35.866366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f3373f-d6a7-4a2e-bea8-3861a29cf8b5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:35.868704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 654/1309] 10.222.0.67 () {54 vars in 964 bytes} [Mon Sep 26 17:54:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:54:35.874985 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a6f7a-587b-48f6-8b46-d125050d7b68 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:35.875346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a6f7a-587b-48f6-8b46-d125050d7b68 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:35.875712 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a6f7a-587b-48f6-8b46-d125050d7b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:35.966780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-141a6f7a-587b-48f6-8b46-d125050d7b68 None None] MFA Rules not processed for user `ef7e0b772a284e7da4fae7d73f34d3de`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:54:35.968097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-141a6f7a-587b-48f6-8b46-d125050d7b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:35.994320 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:36.063044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 656/1310] 10.222.0.67 () {58 vars in 1061 bytes} [Mon Sep 26 17:54:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:54:36.073924 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3373f-d6a7-4a2e-bea8-3861a29cf8b5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:54:36.075793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d2f3373f-d6a7-4a2e-bea8-3861a29cf8b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:36.198558 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f3373f-d6a7-4a2e-bea8-3861a29cf8b5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:54:36.286702 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30617595-5a88-4f4d-8c0f-a4e96e935469 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:54:36.287432 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30617595-5a88-4f4d-8c0f-a4e96e935469 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:36.287799 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30617595-5a88-4f4d-8c0f-a4e96e935469 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:36.288135 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30617595-5a88-4f4d-8c0f-a4e96e935469 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:36.290459 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-30617595-5a88-4f4d-8c0f-a4e96e935469 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:36.352489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:36.420742 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30617595-5a88-4f4d-8c0f-a4e96e935469 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:54:36.422609 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30617595-5a88-4f4d-8c0f-a4e96e935469 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:54:36.444246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 655/1311] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:54:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:54:39.001723 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb9df9-9870-4319-a8d8-e5a42ed86122 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:39.002156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb9df9-9870-4319-a8d8-e5a42ed86122 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:39.002267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb9df9-9870-4319-a8d8-e5a42ed86122 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:39.003600 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 657/1312] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:54:39.009385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955808f-b038-4115-8ada-2a1c291b5547 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:39.009794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955808f-b038-4115-8ada-2a1c291b5547 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:39.009934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955808f-b038-4115-8ada-2a1c291b5547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:39.082605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8955808f-b038-4115-8ada-2a1c291b5547 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:54:39.083835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8955808f-b038-4115-8ada-2a1c291b5547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:39.178392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 656/1313] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:54:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:54:39.472772 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb9df9-9870-4319-a8d8-e5a42ed86122 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:54:39.474312 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-34eb9df9-9870-4319-a8d8-e5a42ed86122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:39.579364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb9df9-9870-4319-a8d8-e5a42ed86122 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:54:39.658034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:39.661307 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac14ad-e52d-40a4-97aa-7c613c0fc880 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:54:39.661920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac14ad-e52d-40a4-97aa-7c613c0fc880 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:39.662200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac14ad-e52d-40a4-97aa-7c613c0fc880 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:39.662471 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac14ad-e52d-40a4-97aa-7c613c0fc880 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:39.664091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-68ac14ad-e52d-40a4-97aa-7c613c0fc880 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:39.770963 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac14ad-e52d-40a4-97aa-7c613c0fc880 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:54:39.772347 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac14ad-e52d-40a4-97aa-7c613c0fc880 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:54:39.791069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 658/1314] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:54:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:54:42.430842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4246bf-8a69-4e32-8c98-6cb87f7c8a29 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:42.431403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4246bf-8a69-4e32-8c98-6cb87f7c8a29 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:42.431403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4246bf-8a69-4e32-8c98-6cb87f7c8a29 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:42.432746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 657/1315] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:54:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:54:42.437864 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f52ec-340a-4a99-b38b-75b454b3a64e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:42.438239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f52ec-340a-4a99-b38b-75b454b3a64e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:42.438410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f52ec-340a-4a99-b38b-75b454b3a64e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:42.504661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a87f52ec-340a-4a99-b38b-75b454b3a64e None None] MFA Rules not processed for user `9b018c400b114999854f81cf97dcfca1`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:54:42.505867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a87f52ec-340a-4a99-b38b-75b454b3a64e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:42.557323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:42.601397 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 659/1316] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:54:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:54:42.888830 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4246bf-8a69-4e32-8c98-6cb87f7c8a29 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:54:42.890472 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2b4246bf-8a69-4e32-8c98-6cb87f7c8a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:42.901127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:43.008663 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4246bf-8a69-4e32-8c98-6cb87f7c8a29 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:54:43.082497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b0205-097d-47bf-906a-8b836528b3f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:54:43.083125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b0205-097d-47bf-906a-8b836528b3f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:43.083407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b0205-097d-47bf-906a-8b836528b3f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:43.083707 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b0205-097d-47bf-906a-8b836528b3f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:43.085320 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-722b0205-097d-47bf-906a-8b836528b3f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:43.147130 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:43.193611 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b0205-097d-47bf-906a-8b836528b3f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:54:43.195039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b0205-097d-47bf-906a-8b836528b3f7 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:54:43.213258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 658/1317] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:54:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:54:46.200805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d927d-4d23-45d8-9826-24795b4435a7 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:46.201368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d927d-4d23-45d8-9826-24795b4435a7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:46.201368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0d927d-4d23-45d8-9826-24795b4435a7 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:46.202698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 660/1318] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:54:46.207949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093d4ad-e32f-4934-855a-c7589bbf4225 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:46.208335 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093d4ad-e32f-4934-855a-c7589bbf4225 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:46.208493 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6093d4ad-e32f-4934-855a-c7589bbf4225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:46.274070 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6093d4ad-e32f-4934-855a-c7589bbf4225 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:54:46.275196 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6093d4ad-e32f-4934-855a-c7589bbf4225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:46.370174 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 659/1319] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:54:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:54:46.657182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d927d-4d23-45d8-9826-24795b4435a7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:54:46.658743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bd0d927d-4d23-45d8-9826-24795b4435a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:46.761823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0d927d-4d23-45d8-9826-24795b4435a7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:54:46.834370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195b3bcb-8751-4c8a-aed3-279be995dfb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:54:46.834970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b3bcb-8751-4c8a-aed3-279be995dfb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:46.835252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b3bcb-8751-4c8a-aed3-279be995dfb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:46.835523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195b3bcb-8751-4c8a-aed3-279be995dfb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:46.837667 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-195b3bcb-8751-4c8a-aed3-279be995dfb2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:46.844509 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:46.946303 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b3bcb-8751-4c8a-aed3-279be995dfb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:54:46.950150 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195b3bcb-8751-4c8a-aed3-279be995dfb2 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:54:46.965867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 661/1320] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:54:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:54:50.306201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb6f93-19a7-4211-8cf9-e862ec7ffa9b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:50.306735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb6f93-19a7-4211-8cf9-e862ec7ffa9b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:50.306735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cb6f93-19a7-4211-8cf9-e862ec7ffa9b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:50.308067 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 660/1321] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:54:50.313626 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa976f-9934-42fa-9514-4acae6697037 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:50.314100 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa976f-9934-42fa-9514-4acae6697037 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:50.314270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa976f-9934-42fa-9514-4acae6697037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:50.382100 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a4fa976f-9934-42fa-9514-4acae6697037 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:54:50.383916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a4fa976f-9934-42fa-9514-4acae6697037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:50.442732 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:50.482964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 662/1322] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:54:50.771114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb6f93-19a7-4211-8cf9-e862ec7ffa9b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:54:50.772684 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b5cb6f93-19a7-4211-8cf9-e862ec7ffa9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:50.793673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:50.878465 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cb6f93-19a7-4211-8cf9-e862ec7ffa9b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:54:50.959749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba41cc-40b2-4954-9add-07c45068e02a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:54:50.959749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba41cc-40b2-4954-9add-07c45068e02a service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:50.959749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba41cc-40b2-4954-9add-07c45068e02a service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:50.959749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba41cc-40b2-4954-9add-07c45068e02a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:50.960215 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9dba41cc-40b2-4954-9add-07c45068e02a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:51.033054 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:51.075499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba41cc-40b2-4954-9add-07c45068e02a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:54:51.077699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba41cc-40b2-4954-9add-07c45068e02a service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:54:51.099680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 661/1323] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:54:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:54:53.398989 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde659e-4ac7-4599-a2fd-b02f9b5f3153 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:53.399527 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde659e-4ac7-4599-a2fd-b02f9b5f3153 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:53.399826 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abde659e-4ac7-4599-a2fd-b02f9b5f3153 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:53.401470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 663/1324] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:54:53.407385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cac52-8233-4b88-8d76-dfbf1989f57a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:53.407875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cac52-8233-4b88-8d76-dfbf1989f57a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:53.407966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28cac52-8233-4b88-8d76-dfbf1989f57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:53.481011 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b28cac52-8233-4b88-8d76-dfbf1989f57a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:54:53.482246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b28cac52-8233-4b88-8d76-dfbf1989f57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:53.584796 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 662/1325] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:54:53.868439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde659e-4ac7-4599-a2fd-b02f9b5f3153 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:54:53.870039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-abde659e-4ac7-4599-a2fd-b02f9b5f3153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:53.978589 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abde659e-4ac7-4599-a2fd-b02f9b5f3153 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:54:54.055512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bddfce-9845-469a-a90f-882b36290256 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:54:54.056153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bddfce-9845-469a-a90f-882b36290256 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:54.056434 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bddfce-9845-469a-a90f-882b36290256 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:54.056710 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bddfce-9845-469a-a90f-882b36290256 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:54.058411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-33bddfce-9845-469a-a90f-882b36290256 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:54.076369 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:54.170782 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bddfce-9845-469a-a90f-882b36290256 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:54:54.172258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bddfce-9845-469a-a90f-882b36290256 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:54:54.191748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 664/1326] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 17:54:56.411589 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66226804-d680-4dc0-ab57-71c8afd0f74e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:56.412054 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66226804-d680-4dc0-ab57-71c8afd0f74e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:56.412224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66226804-d680-4dc0-ab57-71c8afd0f74e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:56.413503 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 663/1327] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:54:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:54:56.420237 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb00f5-7f9b-4d41-b6be-d3f77846204d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:56.420718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb00f5-7f9b-4d41-b6be-d3f77846204d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:56.420813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb00f5-7f9b-4d41-b6be-d3f77846204d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:56.488503 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d2bb00f5-7f9b-4d41-b6be-d3f77846204d None None] MFA Rules not processed for user `9b018c400b114999854f81cf97dcfca1`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:54:56.489714 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d2bb00f5-7f9b-4d41-b6be-d3f77846204d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:56.557863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:56.590643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 665/1328] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:54:56 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:54:56.879835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66226804-d680-4dc0-ab57-71c8afd0f74e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:54:56.881444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-66226804-d680-4dc0-ab57-71c8afd0f74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:56.913420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:57.001821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66226804-d680-4dc0-ab57-71c8afd0f74e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:54:57.091581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc28db3-33d9-4239-b97f-ee53ec7e538e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:54:57.092234 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc28db3-33d9-4239-b97f-ee53ec7e538e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:57.092511 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc28db3-33d9-4239-b97f-ee53ec7e538e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:57.092780 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc28db3-33d9-4239-b97f-ee53ec7e538e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:57.094522 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6bc28db3-33d9-4239-b97f-ee53ec7e538e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:54:57.190153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:54:57.227901 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc28db3-33d9-4239-b97f-ee53ec7e538e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:54:57.229426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc28db3-33d9-4239-b97f-ee53ec7e538e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:54:57.250478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 664/1329] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:54:56 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:54:59.871893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84df275-8f97-46ab-bc7f-e4673b3c9914 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:59.872460 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84df275-8f97-46ab-bc7f-e4673b3c9914 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:59.872460 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84df275-8f97-46ab-bc7f-e4673b3c9914 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:59.873952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 666/1330] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:54:59.879069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb194ab-1ffa-4425-8420-0aeef7d67359 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:54:59.879597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb194ab-1ffa-4425-8420-0aeef7d67359 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:54:59.879597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb194ab-1ffa-4425-8420-0aeef7d67359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:54:59.948363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3bb194ab-1ffa-4425-8420-0aeef7d67359 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:54:59.949594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3bb194ab-1ffa-4425-8420-0aeef7d67359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:00.048082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 665/1331] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:00.350194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f1eec-7161-4ab1-8b17-213590a8a175 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:00.350481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f1eec-7161-4ab1-8b17-213590a8a175 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:00.350758 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f1eec-7161-4ab1-8b17-213590a8a175 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:00.352161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 667/1332] 10.222.0.67 () {54 vars in 964 bytes} [Mon Sep 26 17:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:00.358276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4891b44-e679-4e40-8249-0cd4fb1678b6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:00.358565 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4891b44-e679-4e40-8249-0cd4fb1678b6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:00.358844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4891b44-e679-4e40-8249-0cd4fb1678b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:00.407946 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:00.442731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a4891b44-e679-4e40-8249-0cd4fb1678b6 None None] MFA Rules not processed for user `ef7e0b772a284e7da4fae7d73f34d3de`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:00.443986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a4891b44-e679-4e40-8249-0cd4fb1678b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:00.521645 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 666/1333] 10.222.0.67 () {58 vars in 1061 bytes} [Mon Sep 26 17:55:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:00.531008 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f1eec-7161-4ab1-8b17-213590a8a175 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:00.532577 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fb4f1eec-7161-4ab1-8b17-213590a8a175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:00.641992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f1eec-7161-4ab1-8b17-213590a8a175 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:00.719998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3bdc9-02ce-4c5a-84e1-ed0b2fc74405 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:00.720601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3bdc9-02ce-4c5a-84e1-ed0b2fc74405 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:00.720879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3bdc9-02ce-4c5a-84e1-ed0b2fc74405 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:00.721151 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3bdc9-02ce-4c5a-84e1-ed0b2fc74405 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:00.722739 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-37c3bdc9-02ce-4c5a-84e1-ed0b2fc74405 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:00.757802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:00.861591 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3bdc9-02ce-4c5a-84e1-ed0b2fc74405 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:00.863036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3bdc9-02ce-4c5a-84e1-ed0b2fc74405 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:00.881377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 668/1334] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:55:03.345158 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268195f-9749-42ed-b964-420e0ada6a36 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:03.345685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268195f-9749-42ed-b964-420e0ada6a36 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:03.345685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2268195f-9749-42ed-b964-420e0ada6a36 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:03.346993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 667/1335] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:03.352504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b4006-e545-4d1f-b8f8-0e9adde59b22 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:03.353094 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b4006-e545-4d1f-b8f8-0e9adde59b22 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:03.353094 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b4006-e545-4d1f-b8f8-0e9adde59b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:03.419028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-025b4006-e545-4d1f-b8f8-0e9adde59b22 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:03.420250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-025b4006-e545-4d1f-b8f8-0e9adde59b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:03.493965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:03.517151 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 669/1336] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:03.804838 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268195f-9749-42ed-b964-420e0ada6a36 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:03.806374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2268195f-9749-42ed-b964-420e0ada6a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:03.908708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2268195f-9749-42ed-b964-420e0ada6a36 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:03.936951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:03.985018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b29756-7430-4000-9d06-d7804fb6ffea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:03.985624 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b29756-7430-4000-9d06-d7804fb6ffea service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:03.985900 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b29756-7430-4000-9d06-d7804fb6ffea service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:03.986170 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b29756-7430-4000-9d06-d7804fb6ffea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:03.987777 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e6b29756-7430-4000-9d06-d7804fb6ffea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:04.093812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b29756-7430-4000-9d06-d7804fb6ffea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:04.095160 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b29756-7430-4000-9d06-d7804fb6ffea service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:04.114714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 668/1337] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:55:06.819382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70263457-9f80-4a8f-a8d7-f24d1be632a9 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:06.820099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70263457-9f80-4a8f-a8d7-f24d1be632a9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:06.820099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70263457-9f80-4a8f-a8d7-f24d1be632a9 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:06.821319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 670/1338] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:06.826479 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9847ce-1799-4e4f-acb8-d955f8c132b2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:06.826909 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9847ce-1799-4e4f-acb8-d955f8c132b2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:06.827008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9847ce-1799-4e4f-acb8-d955f8c132b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:06.892801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-db9847ce-1799-4e4f-acb8-d955f8c132b2 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:06.894092 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-db9847ce-1799-4e4f-acb8-d955f8c132b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:06.903411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:06.999593 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 669/1339] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:55:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:07.283904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70263457-9f80-4a8f-a8d7-f24d1be632a9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:07.285438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-70263457-9f80-4a8f-a8d7-f24d1be632a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:07.388635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70263457-9f80-4a8f-a8d7-f24d1be632a9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:07.460765 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5d636-43d0-4001-a196-077ab5ed9751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:07.461363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5d636-43d0-4001-a196-077ab5ed9751 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:07.461643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5d636-43d0-4001-a196-077ab5ed9751 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:07.461929 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5d636-43d0-4001-a196-077ab5ed9751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:07.463487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1dd5d636-43d0-4001-a196-077ab5ed9751 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:07.499274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:07.572044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5d636-43d0-4001-a196-077ab5ed9751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:07.573395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5d636-43d0-4001-a196-077ab5ed9751 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:07.591542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 671/1340] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:55:11.979565 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9752680-1520-4115-81e7-63bd63d53397 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:11.980125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9752680-1520-4115-81e7-63bd63d53397 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:11.980125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9752680-1520-4115-81e7-63bd63d53397 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:11.981494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 670/1341] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:11.988467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ad40a-01ca-4186-a80c-e726c1740c1d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:11.988935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ad40a-01ca-4186-a80c-e726c1740c1d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:11.989034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27ad40a-01ca-4186-a80c-e726c1740c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:12.061577 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a27ad40a-01ca-4186-a80c-e726c1740c1d None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:12.062862 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a27ad40a-01ca-4186-a80c-e726c1740c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:12.155618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:12.167081 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 672/1342] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:12.462594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9752680-1520-4115-81e7-63bd63d53397 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:12.464228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f9752680-1520-4115-81e7-63bd63d53397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:12.569958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9752680-1520-4115-81e7-63bd63d53397 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:12.608839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:12.647564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b28d58-a62f-45b3-9ea6-236c86d9c7d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:12.648539 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b28d58-a62f-45b3-9ea6-236c86d9c7d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:12.648821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b28d58-a62f-45b3-9ea6-236c86d9c7d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:12.649097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b28d58-a62f-45b3-9ea6-236c86d9c7d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:12.650801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-60b28d58-a62f-45b3-9ea6-236c86d9c7d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:12.764668 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b28d58-a62f-45b3-9ea6-236c86d9c7d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:12.766021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b28d58-a62f-45b3-9ea6-236c86d9c7d0 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:12.784792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 671/1343] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:55:16.043021 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9094077-af0b-45bd-980a-b668f4c95e51 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:16.043593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9094077-af0b-45bd-980a-b668f4c95e51 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:16.043593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9094077-af0b-45bd-980a-b668f4c95e51 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:16.044928 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 673/1344] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:16.050393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3df754-1416-4e16-af05-f08c3d847c52 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:16.050820 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3df754-1416-4e16-af05-f08c3d847c52 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:16.050943 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3df754-1416-4e16-af05-f08c3d847c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:16.119059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6e3df754-1416-4e16-af05-f08c3d847c52 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:16.120290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6e3df754-1416-4e16-af05-f08c3d847c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:16.139899 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:16.223903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 672/1345] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:55:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:55:16.508058 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9094077-af0b-45bd-980a-b668f4c95e51 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:16.509585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c9094077-af0b-45bd-980a-b668f4c95e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:16.615135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9094077-af0b-45bd-980a-b668f4c95e51 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:16.714058 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f74663-2727-4977-8d1b-9873e00e2629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:16.714684 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f74663-2727-4977-8d1b-9873e00e2629 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:16.714969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f74663-2727-4977-8d1b-9873e00e2629 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:16.715238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f74663-2727-4977-8d1b-9873e00e2629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:16.716903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-10f74663-2727-4977-8d1b-9873e00e2629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:16.765005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:16.838046 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f74663-2727-4977-8d1b-9873e00e2629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:16.839501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f74663-2727-4977-8d1b-9873e00e2629 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:16.860457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 674/1346] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 17:55:21.340376 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af66252-f5ad-4dfb-b87d-e5f41ef73db7 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:21.345415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af66252-f5ad-4dfb-b87d-e5f41ef73db7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:21.345415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af66252-f5ad-4dfb-b87d-e5f41ef73db7 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:21.345415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 673/1347] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:21.348216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fc099-81cb-4924-9633-74c469c59923 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:21.348728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fc099-81cb-4924-9633-74c469c59923 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:21.348826 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702fc099-81cb-4924-9633-74c469c59923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:21.426830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-702fc099-81cb-4924-9633-74c469c59923 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:21.428146 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-702fc099-81cb-4924-9633-74c469c59923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:21.528779 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:21.531056 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 675/1348] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:55:21.818230 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af66252-f5ad-4dfb-b87d-e5f41ef73db7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:21.820708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6af66252-f5ad-4dfb-b87d-e5f41ef73db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:21.934912 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af66252-f5ad-4dfb-b87d-e5f41ef73db7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:21.983152 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:22.009436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e625eab2-6be5-46c0-9435-4e32811a7dca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:22.010041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625eab2-6be5-46c0-9435-4e32811a7dca service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:22.010324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625eab2-6be5-46c0-9435-4e32811a7dca service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:22.010599 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e625eab2-6be5-46c0-9435-4e32811a7dca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:22.012270 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e625eab2-6be5-46c0-9435-4e32811a7dca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:22.122010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625eab2-6be5-46c0-9435-4e32811a7dca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:22.123428 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e625eab2-6be5-46c0-9435-4e32811a7dca service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:22.142179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 674/1349] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:55:27.158288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d363c8-047f-4087-98c5-8300bcc8ba60 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:27.158780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d363c8-047f-4087-98c5-8300bcc8ba60 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:27.158863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d363c8-047f-4087-98c5-8300bcc8ba60 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:27.160575 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 676/1350] 10.222.0.67 () {54 vars in 921 bytes} [Mon Sep 26 17:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:27.165692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305891a7-30ac-4e9a-b734-e60a14b7e086 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:27.166098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305891a7-30ac-4e9a-b734-e60a14b7e086 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:27.166553 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305891a7-30ac-4e9a-b734-e60a14b7e086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:27.319531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-305891a7-30ac-4e9a-b734-e60a14b7e086 None None] MFA Rules not processed for user `22a512e0a64e420abf15b2150ccc2b32`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:27.320922 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-305891a7-30ac-4e9a-b734-e60a14b7e086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:27.340904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:27.452788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 675/1351] 10.222.0.67 () {58 vars in 1018 bytes} [Mon Sep 26 17:55:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:31.670423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b060af-3114-49c0-943c-a6db715dd24a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:31.670933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b060af-3114-49c0-943c-a6db715dd24a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:31.670933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b060af-3114-49c0-943c-a6db715dd24a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:31.672494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 677/1352] 10.222.0.67 () {54 vars in 921 bytes} [Mon Sep 26 17:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:31.677464 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1630fd-2ad8-4ac8-9d43-1f7ddb654797 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:31.678448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1630fd-2ad8-4ac8-9d43-1f7ddb654797 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:31.678623 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1630fd-2ad8-4ac8-9d43-1f7ddb654797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:31.758854 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7a1630fd-2ad8-4ac8-9d43-1f7ddb654797 None None] MFA Rules not processed for user `22a512e0a64e420abf15b2150ccc2b32`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:31.760094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7a1630fd-2ad8-4ac8-9d43-1f7ddb654797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:31.840241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:31.882643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 676/1353] 10.222.0.67 () {58 vars in 1018 bytes} [Mon Sep 26 17:55:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:33.282195 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4098f87-aa4c-4bcc-ab75-390a8e445ab7 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:33.282719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4098f87-aa4c-4bcc-ab75-390a8e445ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:33.282719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4098f87-aa4c-4bcc-ab75-390a8e445ab7 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:33.284000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 678/1354] 10.222.0.67 () {54 vars in 921 bytes} [Mon Sep 26 17:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:33.290122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f104e1-b39f-448a-afbc-478bb35a10ab None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:33.290577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f104e1-b39f-448a-afbc-478bb35a10ab None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:33.290830 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f104e1-b39f-448a-afbc-478bb35a10ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:33.370967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-57f104e1-b39f-448a-afbc-478bb35a10ab None None] MFA Rules not processed for user `22a512e0a64e420abf15b2150ccc2b32`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:33.372213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-57f104e1-b39f-448a-afbc-478bb35a10ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:33.485652 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 677/1355] 10.222.0.67 () {58 vars in 1018 bytes} [Mon Sep 26 17:55:33 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:37.000029 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dca46-0e5c-4b95-9712-e48aa2491e83 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:37.000663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dca46-0e5c-4b95-9712-e48aa2491e83 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:37.000663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26dca46-0e5c-4b95-9712-e48aa2491e83 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:37.001889 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 679/1356] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:37.008648 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cd92a-dc41-4497-b0b4-1243f4d1a5f2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:37.009226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cd92a-dc41-4497-b0b4-1243f4d1a5f2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:37.009226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727cd92a-dc41-4497-b0b4-1243f4d1a5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:37.050863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:37.086910 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-727cd92a-dc41-4497-b0b4-1243f4d1a5f2 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:37.088241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-727cd92a-dc41-4497-b0b4-1243f4d1a5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:37.197228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 678/1357] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:55:37.518494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319db2fe-1da7-482c-876e-445b416bc9d8 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:37.519285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319db2fe-1da7-482c-876e-445b416bc9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:37.519931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319db2fe-1da7-482c-876e-445b416bc9d8 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:37.523609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 680/1358] 10.222.0.67 () {54 vars in 961 bytes} [Mon Sep 26 17:55:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:37.531359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07330e-a7b1-4f29-9fe1-7de4d1d795df None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:37.539773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07330e-a7b1-4f29-9fe1-7de4d1d795df None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:37.539773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07330e-a7b1-4f29-9fe1-7de4d1d795df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:37.622626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5a07330e-a7b1-4f29-9fe1-7de4d1d795df None None] MFA Rules not processed for user `b9ed29a3f6be4d64bb3751f9e95eb56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:37.625035 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5a07330e-a7b1-4f29-9fe1-7de4d1d795df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:37.679590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:37.750482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 679/1359] 10.222.0.67 () {58 vars in 1058 bytes} [Mon Sep 26 17:55:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:37.759239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319db2fe-1da7-482c-876e-445b416bc9d8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:37.760781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-319db2fe-1da7-482c-876e-445b416bc9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:37.918183 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319db2fe-1da7-482c-876e-445b416bc9d8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:38.003554 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:38.047854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54d9e8d-d456-4aef-9782-bca9f9b05274 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:38.048460 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d9e8d-d456-4aef-9782-bca9f9b05274 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:38.048772 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d9e8d-d456-4aef-9782-bca9f9b05274 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:38.049079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54d9e8d-d456-4aef-9782-bca9f9b05274 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:38.050741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b54d9e8d-d456-4aef-9782-bca9f9b05274 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:38.160587 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d9e8d-d456-4aef-9782-bca9f9b05274 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:38.162284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54d9e8d-d456-4aef-9782-bca9f9b05274 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:38.184128 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 681/1360] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:55:40.595971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d17c13-eb50-4e2e-b090-a1bbc494747a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:40.596475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d17c13-eb50-4e2e-b090-a1bbc494747a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:40.596475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d17c13-eb50-4e2e-b090-a1bbc494747a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:40.598953 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 680/1361] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:40.603617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee7d26-9da4-4a42-86e6-ea8bf2eb219c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:40.604134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee7d26-9da4-4a42-86e6-ea8bf2eb219c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:40.604292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee7d26-9da4-4a42-86e6-ea8bf2eb219c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:40.675892 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-acee7d26-9da4-4a42-86e6-ea8bf2eb219c None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:40.677109 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-acee7d26-9da4-4a42-86e6-ea8bf2eb219c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:40.718279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:40.780672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 682/1362] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:41.074892 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d17c13-eb50-4e2e-b090-a1bbc494747a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:41.076604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e8d17c13-eb50-4e2e-b090-a1bbc494747a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:41.260058 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d17c13-eb50-4e2e-b090-a1bbc494747a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:41.270589 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:41.385022 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33eb042-5834-4bed-9e65-1caa42494b40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:41.385639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33eb042-5834-4bed-9e65-1caa42494b40 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:41.385917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33eb042-5834-4bed-9e65-1caa42494b40 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:41.386189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33eb042-5834-4bed-9e65-1caa42494b40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:41.392275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b33eb042-5834-4bed-9e65-1caa42494b40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:41.491921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33eb042-5834-4bed-9e65-1caa42494b40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:41.493538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33eb042-5834-4bed-9e65-1caa42494b40 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:41.511455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 681/1363] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:55:44.208170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dab86-e8bb-4d90-85f4-32ba5e706dd1 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:44.208731 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dab86-e8bb-4d90-85f4-32ba5e706dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:44.209176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245dab86-e8bb-4d90-85f4-32ba5e706dd1 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:44.211625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 683/1364] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:55:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:44.217078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9dd08-ad50-41c3-8d26-8ca03b674506 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:44.217513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9dd08-ad50-41c3-8d26-8ca03b674506 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:44.217688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df9dd08-ad50-41c3-8d26-8ca03b674506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:44.234678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:44.289740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5df9dd08-ad50-41c3-8d26-8ca03b674506 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:44.290871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5df9dd08-ad50-41c3-8d26-8ca03b674506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:44.395304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 682/1365] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:55:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:44.689832 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48be606-064f-4e92-b44d-df7578215133 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:44.690433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48be606-064f-4e92-b44d-df7578215133 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:44.690814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48be606-064f-4e92-b44d-df7578215133 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:44.692462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 684/1366] 10.222.0.67 () {54 vars in 961 bytes} [Mon Sep 26 17:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:44.698576 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c8fed-4909-4e05-924d-0e0862828d5c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:44.698864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c8fed-4909-4e05-924d-0e0862828d5c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:44.704149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4c8fed-4909-4e05-924d-0e0862828d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:44.779728 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1a4c8fed-4909-4e05-924d-0e0862828d5c None None] MFA Rules not processed for user `b9ed29a3f6be4d64bb3751f9e95eb56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:44.780939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1a4c8fed-4909-4e05-924d-0e0862828d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:44.819300 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:44.904785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 683/1367] 10.222.0.67 () {58 vars in 1058 bytes} [Mon Sep 26 17:55:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:55:44.913754 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48be606-064f-4e92-b44d-df7578215133 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:44.915306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c48be606-064f-4e92-b44d-df7578215133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:45.080754 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48be606-064f-4e92-b44d-df7578215133 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:45.105582 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:45.216312 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d2c3ab-0277-4f5f-8e13-924a19bfa6a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:45.216972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2c3ab-0277-4f5f-8e13-924a19bfa6a9 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:45.217210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2c3ab-0277-4f5f-8e13-924a19bfa6a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:45.217425 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d2c3ab-0277-4f5f-8e13-924a19bfa6a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:45.218964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-02d2c3ab-0277-4f5f-8e13-924a19bfa6a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:45.222261 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c403a-2677-4c9e-9af0-b28554ea6032 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:45.222723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c403a-2677-4c9e-9af0-b28554ea6032 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:45.222723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c403a-2677-4c9e-9af0-b28554ea6032 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:45.224281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 684/1368] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:45.229488 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c5e40-3f75-4033-9bc4-fa5e734598ea None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:45.229772 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c5e40-3f75-4033-9bc4-fa5e734598ea None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:45.230047 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c5e40-3f75-4033-9bc4-fa5e734598ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:45.309782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-453c5e40-3f75-4033-9bc4-fa5e734598ea None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:45.310997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-453c5e40-3f75-4033-9bc4-fa5e734598ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:45.319225 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2c3ab-0277-4f5f-8e13-924a19bfa6a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:45.320853 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d2c3ab-0277-4f5f-8e13-924a19bfa6a9 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:45.341273 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 685/1369] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 17:55:45.416938 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:45.424637 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 685/1370] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:55:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:55:45.774422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c134d1-416c-419f-a46a-1b514e26894e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:45.774422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c134d1-416c-419f-a46a-1b514e26894e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:45.774422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c134d1-416c-419f-a46a-1b514e26894e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:45.776149 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 686/1371] 10.222.0.67 () {54 vars in 963 bytes} [Mon Sep 26 17:55:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:45.781501 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6839d-e9eb-44f9-8093-52a4b66c823a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:45.781791 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6839d-e9eb-44f9-8093-52a4b66c823a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:45.782738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a6839d-e9eb-44f9-8093-52a4b66c823a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:45.863867 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-63a6839d-e9eb-44f9-8093-52a4b66c823a None None] MFA Rules not processed for user `e6d98d51966f48d3a1b22f9b1c7963be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:45.865279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-63a6839d-e9eb-44f9-8093-52a4b66c823a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:45.951445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 686/1372] 10.222.0.67 () {58 vars in 1060 bytes} [Mon Sep 26 17:55:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:45.959115 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c134d1-416c-419f-a46a-1b514e26894e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:45.961370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b3c134d1-416c-419f-a46a-1b514e26894e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:45.993232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:46.074758 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c134d1-416c-419f-a46a-1b514e26894e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:46.155749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae335d21-07aa-4cfd-8361-efd92e93614a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:46.156365 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae335d21-07aa-4cfd-8361-efd92e93614a service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:46.156836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae335d21-07aa-4cfd-8361-efd92e93614a service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:46.156950 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae335d21-07aa-4cfd-8361-efd92e93614a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:46.158507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ae335d21-07aa-4cfd-8361-efd92e93614a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:46.245718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:46.281121 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae335d21-07aa-4cfd-8361-efd92e93614a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:46.283268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae335d21-07aa-4cfd-8361-efd92e93614a service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:46.311972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 687/1373] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 17:55:47.741132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fec5b-070c-46b7-a51e-366276cc0dc0 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:47.741702 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fec5b-070c-46b7-a51e-366276cc0dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:47.741702 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3fec5b-070c-46b7-a51e-366276cc0dc0 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:47.743009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 687/1374] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:55:47.748503 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e40ba-4964-4efc-b5d3-b82c83add2bb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:47.748964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e40ba-4964-4efc-b5d3-b82c83add2bb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:47.749179 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2e40ba-4964-4efc-b5d3-b82c83add2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:47.818359 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0a2e40ba-4964-4efc-b5d3-b82c83add2bb None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:47.819653 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0a2e40ba-4964-4efc-b5d3-b82c83add2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:47.917745 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 688/1375] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:55:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:55:48.213162 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3fec5b-070c-46b7-a51e-366276cc0dc0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:48.214820 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fc3fec5b-070c-46b7-a51e-366276cc0dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:48.306372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:48.371568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3fec5b-070c-46b7-a51e-366276cc0dc0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:48.496187 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe2af0-e0dd-442b-b884-c3c9cbae90e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:48.496930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe2af0-e0dd-442b-b884-c3c9cbae90e6 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:48.497206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe2af0-e0dd-442b-b884-c3c9cbae90e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:48.497474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe2af0-e0dd-442b-b884-c3c9cbae90e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:48.499072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9bbe2af0-e0dd-442b-b884-c3c9cbae90e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:48.551970 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:48.595902 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe2af0-e0dd-442b-b884-c3c9cbae90e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:48.597526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe2af0-e0dd-442b-b884-c3c9cbae90e6 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:48.618491 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 688/1376] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:55:50.984254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd382c-f032-4515-97a4-a78843698115 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:50.984718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd382c-f032-4515-97a4-a78843698115 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:50.984800 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdd382c-f032-4515-97a4-a78843698115 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:50.986134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 689/1377] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:55:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:50.992309 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8d4b4-0206-40d5-98db-eb8f68774dc4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:50.996978 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8d4b4-0206-40d5-98db-eb8f68774dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:50.996978 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b8d4b4-0206-40d5-98db-eb8f68774dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:51.063347 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-11b8d4b4-0206-40d5-98db-eb8f68774dc4 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:51.064592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-11b8d4b4-0206-40d5-98db-eb8f68774dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:51.178665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 689/1378] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:51.470702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd382c-f032-4515-97a4-a78843698115 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:51.472234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ccdd382c-f032-4515-97a4-a78843698115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:51.522546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:51.643200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdd382c-f032-4515-97a4-a78843698115 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:51.761256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:51.767169 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95618c-99d5-409a-a7a5-245b746d00dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:51.767911 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95618c-99d5-409a-a7a5-245b746d00dd service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:51.768253 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95618c-99d5-409a-a7a5-245b746d00dd service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:51.774919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95618c-99d5-409a-a7a5-245b746d00dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:51.774919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8e95618c-99d5-409a-a7a5-245b746d00dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:51.867848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95618c-99d5-409a-a7a5-245b746d00dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:51.869437 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95618c-99d5-409a-a7a5-245b746d00dd service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:51.891112 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 690/1379] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:55:54.248253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7f816-d71a-4279-a1ab-aaf6a584ceaa None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:54.249299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7f816-d71a-4279-a1ab-aaf6a584ceaa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:54.249695 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c7f816-d71a-4279-a1ab-aaf6a584ceaa None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:54.251105 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 690/1380] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:55:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:54.257526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359fcd7-f997-457e-aba2-dd8ec3593d49 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:54.258046 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359fcd7-f997-457e-aba2-dd8ec3593d49 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:54.258216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c359fcd7-f997-457e-aba2-dd8ec3593d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:54.327544 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c359fcd7-f997-457e-aba2-dd8ec3593d49 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:54.328801 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c359fcd7-f997-457e-aba2-dd8ec3593d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:54.404109 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:54.444450 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 691/1381] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:54.737058 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7f816-d71a-4279-a1ab-aaf6a584ceaa None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:54.738590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e0c7f816-d71a-4279-a1ab-aaf6a584ceaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:54.768182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:54.890601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c7f816-d71a-4279-a1ab-aaf6a584ceaa None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:55.009389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:55.031187 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536c695d-e1e7-4e93-920c-cada49b568cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:55.032059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c695d-e1e7-4e93-920c-cada49b568cf service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:55.032450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c695d-e1e7-4e93-920c-cada49b568cf service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:55.032850 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c695d-e1e7-4e93-920c-cada49b568cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:55.035120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-536c695d-e1e7-4e93-920c-cada49b568cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:55.146573 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c695d-e1e7-4e93-920c-cada49b568cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:55.148666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536c695d-e1e7-4e93-920c-cada49b568cf service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:55.173733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 691/1382] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:55:58.168900 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c096a-13ad-4b54-b409-d495f3815a5a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:58.169427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c096a-13ad-4b54-b409-d495f3815a5a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:58.169603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c096a-13ad-4b54-b409-d495f3815a5a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:58.171311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 692/1383] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:55:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:55:58.179977 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b23ea6-7292-4c81-a9e4-c8d7bafb34aa None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:58.180514 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b23ea6-7292-4c81-a9e4-c8d7bafb34aa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:58.180664 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b23ea6-7292-4c81-a9e4-c8d7bafb34aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:58.267152 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d9b23ea6-7292-4c81-a9e4-c8d7bafb34aa None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:55:58.268326 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d9b23ea6-7292-4c81-a9e4-c8d7bafb34aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:58.335949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:58.378144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 692/1384] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:55:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:55:58.777665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c096a-13ad-4b54-b409-d495f3815a5a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:55:58.779207 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-102c096a-13ad-4b54-b409-d495f3815a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:58.933167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c096a-13ad-4b54-b409-d495f3815a5a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:55:58.999593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:55:59.066367 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a38e84-f624-4c31-b8d4-651e9962bc1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:55:59.066937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a38e84-f624-4c31-b8d4-651e9962bc1c service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:55:59.067228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a38e84-f624-4c31-b8d4-651e9962bc1c service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:55:59.067523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a38e84-f624-4c31-b8d4-651e9962bc1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:55:59.069265 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-14a38e84-f624-4c31-b8d4-651e9962bc1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:55:59.171355 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a38e84-f624-4c31-b8d4-651e9962bc1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:55:59.173124 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a38e84-f624-4c31-b8d4-651e9962bc1c service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:55:59.193692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 693/1385] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:55:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 17:56:01.475996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a04c53-f9b6-4e13-ace7-ebf240823054 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:01.476696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a04c53-f9b6-4e13-ace7-ebf240823054 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:01.476696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a04c53-f9b6-4e13-ace7-ebf240823054 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:01.477926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 693/1386] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:01.483229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095db20-5fd9-4038-a692-dad66484130e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:01.483673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095db20-5fd9-4038-a692-dad66484130e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:01.483810 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095db20-5fd9-4038-a692-dad66484130e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:01.551511 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2095db20-5fd9-4038-a692-dad66484130e None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:01.553923 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2095db20-5fd9-4038-a692-dad66484130e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:01.562337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:01.667504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 694/1387] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:56:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:01.960241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a04c53-f9b6-4e13-ace7-ebf240823054 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:01.961753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-09a04c53-f9b6-4e13-ace7-ebf240823054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:02.126230 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a04c53-f9b6-4e13-ace7-ebf240823054 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:02.127734 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f556e37-72db-45d3-87ab-d73e2bfc170a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:02.128026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f556e37-72db-45d3-87ab-d73e2bfc170a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:02.128299 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f556e37-72db-45d3-87ab-d73e2bfc170a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:02.130368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 695/1388] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:56:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:56:02.136063 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e2a8c-a441-4d09-a972-6702f51e2f65 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:02.136859 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e2a8c-a441-4d09-a972-6702f51e2f65 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:02.137151 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e2a8c-a441-4d09-a972-6702f51e2f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:02.177765 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:02.205161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0a5e2a8c-a441-4d09-a972-6702f51e2f65 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:02.206383 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0a5e2a8c-a441-4d09-a972-6702f51e2f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:02.249014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d8b55-cfe2-4811-9b84-558a8c1f556c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:02.249634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d8b55-cfe2-4811-9b84-558a8c1f556c service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:02.249918 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d8b55-cfe2-4811-9b84-558a8c1f556c service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:02.250219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d8b55-cfe2-4811-9b84-558a8c1f556c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:02.251921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-917d8b55-cfe2-4811-9b84-558a8c1f556c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:02.294446 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:02.305336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 696/1389] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:56:02.347986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d8b55-cfe2-4811-9b84-558a8c1f556c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:02.349713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d8b55-cfe2-4811-9b84-558a8c1f556c service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:02.368266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 694/1390] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:02.614088 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a9f92-1abe-4003-a0ba-a0513e24bfa5 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:02.615352 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a9f92-1abe-4003-a0ba-a0513e24bfa5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:02.615352 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a9f92-1abe-4003-a0ba-a0513e24bfa5 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:02.616697 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 697/1391] 10.222.0.67 () {54 vars in 962 bytes} [Mon Sep 26 17:56:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:02.622813 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc4901-756b-47ec-9a27-995687f62958 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:02.623385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc4901-756b-47ec-9a27-995687f62958 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:02.623385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc4901-756b-47ec-9a27-995687f62958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:02.687228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:02.708869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-54bc4901-756b-47ec-9a27-995687f62958 None None] MFA Rules not processed for user `e6dee005fc3d4fb593a1771e7851c20a`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:02.710187 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-54bc4901-756b-47ec-9a27-995687f62958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:02.788178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 695/1392] 10.222.0.67 () {58 vars in 1059 bytes} [Mon Sep 26 17:56:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:02.797120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a9f92-1abe-4003-a0ba-a0513e24bfa5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:02.798610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7d1a9f92-1abe-4003-a0ba-a0513e24bfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:02.904775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a9f92-1abe-4003-a0ba-a0513e24bfa5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:02.985252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d08d6-0db0-4b3c-be5b-109716caed11 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6dee005fc3d4fb593a1771e7851c20a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:02.985873 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d08d6-0db0-4b3c-be5b-109716caed11 service glance] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:02.986152 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d08d6-0db0-4b3c-be5b-109716caed11 service glance] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:02.986422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d08d6-0db0-4b3c-be5b-109716caed11 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:02.988155 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-287d08d6-0db0-4b3c-be5b-109716caed11 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:03.036381 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:03.106152 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d08d6-0db0-4b3c-be5b-109716caed11 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:03.107588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287d08d6-0db0-4b3c-be5b-109716caed11 service glance] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:03.126582 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 698/1393] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:04.035207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949de2dc-818f-4cb4-866e-7e356ef01a82 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:04.035688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949de2dc-818f-4cb4-866e-7e356ef01a82 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:04.035958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949de2dc-818f-4cb4-866e-7e356ef01a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:04.106738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-949de2dc-818f-4cb4-866e-7e356ef01a82 None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:04.108122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-949de2dc-818f-4cb4-866e-7e356ef01a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:04.200830 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 696/1394] 10.222.0.67 () {58 vars in 1009 bytes} [Mon Sep 26 17:56:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:04.249187 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c96784-f77d-4faa-b0e4-2ec1d840c030 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:04.249671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c96784-f77d-4faa-b0e4-2ec1d840c030 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:04.249776 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c96784-f77d-4faa-b0e4-2ec1d840c030 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:04.251245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 699/1395] 10.222.0.67 () {54 vars in 943 bytes} [Mon Sep 26 17:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:04.258903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859d022-4010-4c4d-b495-11998c7c4d64 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:04.259201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859d022-4010-4c4d-b495-11998c7c4d64 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:04.259476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7859d022-4010-4c4d-b495-11998c7c4d64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:04.270256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:04.340402 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7859d022-4010-4c4d-b495-11998c7c4d64 None None] MFA Rules not processed for user `7ba12b94717a4a40b55635192840d3c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:04.341611 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7859d022-4010-4c4d-b495-11998c7c4d64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:04.464796 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 697/1396] 10.222.0.67 () {58 vars in 1040 bytes} [Mon Sep 26 17:56:04 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:04.474839 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c96784-f77d-4faa-b0e4-2ec1d840c030 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:04.476420 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-34c96784-f77d-4faa-b0e4-2ec1d840c030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:04.622612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c96784-f77d-4faa-b0e4-2ec1d840c030 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:04.636321 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:04.698896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4ca40-11af-4ded-9884-865564e9a02b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:04.699263 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4ca40-11af-4ded-9884-865564e9a02b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:04.699491 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4ca40-11af-4ded-9884-865564e9a02b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:04.700869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 698/1397] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:56:04.705938 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9704cb1-db10-495c-a757-99959be219ca None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:04.706216 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9704cb1-db10-495c-a757-99959be219ca None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:04.706490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9704cb1-db10-495c-a757-99959be219ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:04.731435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d8e42-c63c-4957-aebe-be7bae97f0d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:04.731971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d8e42-c63c-4957-aebe-be7bae97f0d1 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:04.732247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d8e42-c63c-4957-aebe-be7bae97f0d1 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:04.732516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674d8e42-c63c-4957-aebe-be7bae97f0d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:04.734721 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-674d8e42-c63c-4957-aebe-be7bae97f0d1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:04.773896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a9704cb1-db10-495c-a757-99959be219ca None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:04.775101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a9704cb1-db10-495c-a757-99959be219ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:04.793234 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:04.820003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d8e42-c63c-4957-aebe-be7bae97f0d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:04.821609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674d8e42-c63c-4957-aebe-be7bae97f0d1 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:04.829630 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 700/1398] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 17:56:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 17:56:04.876110 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 699/1399] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:56:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:56:05.167961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d8e42-c63c-4957-aebe-be7bae97f0d1 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:05.169164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-674d8e42-c63c-4957-aebe-be7bae97f0d1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:05.217906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:05.262911 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674d8e42-c63c-4957-aebe-be7bae97f0d1 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:05.411656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bf6711-45a4-4570-93d6-d9bd425d1f02 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:05.412300 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf6711-45a4-4570-93d6-d9bd425d1f02 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:05.412582 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf6711-45a4-4570-93d6-d9bd425d1f02 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:05.412854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bf6711-45a4-4570-93d6-d9bd425d1f02 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:05.414632 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f2bf6711-45a4-4570-93d6-d9bd425d1f02 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:05.487958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:05.520284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf6711-45a4-4570-93d6-d9bd425d1f02 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:05.522695 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bf6711-45a4-4570-93d6-d9bd425d1f02 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:05.542564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 701/1400] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:07.928866 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa74bf-93cc-48fd-88b6-90478e562642 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:07.929498 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa74bf-93cc-48fd-88b6-90478e562642 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:07.929498 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa74bf-93cc-48fd-88b6-90478e562642 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:07.930973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 700/1401] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 17:56:07.936483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee494b-7fa7-495d-930a-3adbf1317834 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:07.937895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee494b-7fa7-495d-930a-3adbf1317834 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:07.937895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee494b-7fa7-495d-930a-3adbf1317834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:08.019276 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-63ee494b-7fa7-495d-930a-3adbf1317834 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:08.020920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-63ee494b-7fa7-495d-930a-3adbf1317834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:08.144038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 702/1402] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:08.457344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa74bf-93cc-48fd-88b6-90478e562642 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:08.459410 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cefa74bf-93cc-48fd-88b6-90478e562642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:08.621592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:08.623033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa74bf-93cc-48fd-88b6-90478e562642 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:08.753448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47194e8-cc70-48ec-bf86-3feca40607e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:08.753961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47194e8-cc70-48ec-bf86-3feca40607e8 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:08.754236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47194e8-cc70-48ec-bf86-3feca40607e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:08.754534 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47194e8-cc70-48ec-bf86-3feca40607e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:08.756243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e47194e8-cc70-48ec-bf86-3feca40607e8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:08.855882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47194e8-cc70-48ec-bf86-3feca40607e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:08.857620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47194e8-cc70-48ec-bf86-3feca40607e8 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:08.877098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 701/1403] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:11.277558 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84a1f8-962e-4513-9333-e15429091c81 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:11.278079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84a1f8-962e-4513-9333-e15429091c81 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:11.278079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84a1f8-962e-4513-9333-e15429091c81 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:11.279312 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 703/1404] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:11.284507 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0479a22-bf81-40f3-af4d-9e8aef0ecfb3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:11.284929 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0479a22-bf81-40f3-af4d-9e8aef0ecfb3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:11.285024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0479a22-bf81-40f3-af4d-9e8aef0ecfb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:11.291126 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:11.356936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f0479a22-bf81-40f3-af4d-9e8aef0ecfb3 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:11.358154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f0479a22-bf81-40f3-af4d-9e8aef0ecfb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:11.467444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 702/1405] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:11.782519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84a1f8-962e-4513-9333-e15429091c81 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:11.784014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ae84a1f8-962e-4513-9333-e15429091c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:11.850562 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:12.005522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae84a1f8-962e-4513-9333-e15429091c81 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:12.138798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c170ce28-d493-44d1-8211-faa8f1cc4c3c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:12.139416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170ce28-d493-44d1-8211-faa8f1cc4c3c service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:12.139721 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170ce28-d493-44d1-8211-faa8f1cc4c3c service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:12.139997 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c170ce28-d493-44d1-8211-faa8f1cc4c3c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:12.141605 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c170ce28-d493-44d1-8211-faa8f1cc4c3c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:12.144056 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:12.253374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170ce28-d493-44d1-8211-faa8f1cc4c3c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:12.255684 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c170ce28-d493-44d1-8211-faa8f1cc4c3c service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:12.276259 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 704/1406] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:14.702561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bfcf4-0492-4d94-8594-356cb6f49285 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:14.703525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bfcf4-0492-4d94-8594-356cb6f49285 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:14.703831 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557bfcf4-0492-4d94-8594-356cb6f49285 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:14.705191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 703/1407] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:56:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:14.711064 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd38f3-0e06-40dd-a511-5ca0babf23a5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:14.711525 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd38f3-0e06-40dd-a511-5ca0babf23a5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:14.711631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dd38f3-0e06-40dd-a511-5ca0babf23a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:14.786829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d2dd38f3-0e06-40dd-a511-5ca0babf23a5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:14.788305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d2dd38f3-0e06-40dd-a511-5ca0babf23a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:14.868250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:14.896581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 705/1408] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:15.196242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bfcf4-0492-4d94-8594-356cb6f49285 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:15.197925 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-557bfcf4-0492-4d94-8594-356cb6f49285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:15.295086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:15.373524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557bfcf4-0492-4d94-8594-356cb6f49285 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:15.499434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8633081d-dbc8-42c1-abba-8c48552a4e35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:15.500248 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633081d-dbc8-42c1-abba-8c48552a4e35 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:15.500630 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633081d-dbc8-42c1-abba-8c48552a4e35 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:15.501010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8633081d-dbc8-42c1-abba-8c48552a4e35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:15.502723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8633081d-dbc8-42c1-abba-8c48552a4e35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:15.556328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:15.602794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633081d-dbc8-42c1-abba-8c48552a4e35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:15.604476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8633081d-dbc8-42c1-abba-8c48552a4e35 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:15.623852 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 704/1409] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:24.903870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=9 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:25.621486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=25 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:29.241607 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c94375-66a9-4a5e-b709-943f59659424 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:29.242141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c94375-66a9-4a5e-b709-943f59659424 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:29.242141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c94375-66a9-4a5e-b709-943f59659424 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:29.243446 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 706/1410] 10.222.0.67 () {54 vars in 918 bytes} [Mon Sep 26 17:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:29.248593 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43bd45-6900-4454-a4ce-509a48cdabcc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:29.249358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43bd45-6900-4454-a4ce-509a48cdabcc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:29.249638 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43bd45-6900-4454-a4ce-509a48cdabcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:29.325505 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8d43bd45-6900-4454-a4ce-509a48cdabcc None None] MFA Rules not processed for user `22a512e0a64e420abf15b2150ccc2b32`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:29.326703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8d43bd45-6900-4454-a4ce-509a48cdabcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:29.433267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 705/1411] 10.222.0.67 () {58 vars in 1015 bytes} [Mon Sep 26 17:56:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:29.448089 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed697a1-40a3-4d63-993d-1eca639b2d4b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:29.448371 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed697a1-40a3-4d63-993d-1eca639b2d4b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:29.448641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed697a1-40a3-4d63-993d-1eca639b2d4b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:29.449953 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 707/1412] 10.222.0.67 () {54 vars in 918 bytes} [Mon Sep 26 17:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:29.454695 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffd1e6-0896-43bc-b53a-5b97724b155e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:29.454975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffd1e6-0896-43bc-b53a-5b97724b155e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:29.455241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dffd1e6-0896-43bc-b53a-5b97724b155e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:29.528019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:29.530819 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2dffd1e6-0896-43bc-b53a-5b97724b155e None None] MFA Rules not processed for user `b9ed29a3f6be4d64bb3751f9e95eb56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:29.532074 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2dffd1e6-0896-43bc-b53a-5b97724b155e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:29.647588 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 706/1413] 10.222.0.67 () {58 vars in 1015 bytes} [Mon Sep 26 17:56:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:56:32.239591 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9790968-7c99-4636-90db-e3a97f1ca28b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:32.240163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9790968-7c99-4636-90db-e3a97f1ca28b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:32.240163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9790968-7c99-4636-90db-e3a97f1ca28b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:32.241574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:56:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:32.246846 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39192f90-8640-4cbd-893a-4ee1cdb89b9c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:32.247280 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39192f90-8640-4cbd-893a-4ee1cdb89b9c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:32.247419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39192f90-8640-4cbd-893a-4ee1cdb89b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:32.314220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-39192f90-8640-4cbd-893a-4ee1cdb89b9c None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:32.315416 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-39192f90-8640-4cbd-893a-4ee1cdb89b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:32.371779 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:32.414826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:56:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:32.710412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9790968-7c99-4636-90db-e3a97f1ca28b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:32.712161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a9790968-7c99-4636-90db-e3a97f1ca28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:32.869315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9790968-7c99-4636-90db-e3a97f1ca28b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:32.943162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:32.984257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b55fd2c-8838-48d2-ad8c-8c05517178be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:32.985007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55fd2c-8838-48d2-ad8c-8c05517178be service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:32.985103 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55fd2c-8838-48d2-ad8c-8c05517178be service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:32.985326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b55fd2c-8838-48d2-ad8c-8c05517178be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:32.986926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6b55fd2c-8838-48d2-ad8c-8c05517178be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:33.080309 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b55fd2c-8838-48d2-ad8c-8c05517178be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:33.082713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b55fd2c-8838-48d2-ad8c-8c05517178be service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:33.101316 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 709/1416] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:34.249273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766eefe-a382-4ad8-82d1-3b2ceb132dfe None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:34.249821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766eefe-a382-4ad8-82d1-3b2ceb132dfe None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:34.249821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766eefe-a382-4ad8-82d1-3b2ceb132dfe None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:34.251176 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:34.258687 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4da231-d744-4aa2-8a75-e679c8fb6b89 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:34.260249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4da231-d744-4aa2-8a75-e679c8fb6b89 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:34.260249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4da231-d744-4aa2-8a75-e679c8fb6b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:34.337461 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5e4da231-d744-4aa2-8a75-e679c8fb6b89 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:34.338706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5e4da231-d744-4aa2-8a75-e679c8fb6b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:34.365688 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:34.436739 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:56:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:56:34.586694 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766eefe-a382-4ad8-82d1-3b2ceb132dfe None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:34.588206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c766eefe-a382-4ad8-82d1-3b2ceb132dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:34.734995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766eefe-a382-4ad8-82d1-3b2ceb132dfe None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:34.776096 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:34.849170 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b46b2c4-ce09-4ef4-b0f4-933787c2e3e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:34.849703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46b2c4-ce09-4ef4-b0f4-933787c2e3e6 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:34.849981 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46b2c4-ce09-4ef4-b0f4-933787c2e3e6 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:34.850251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b46b2c4-ce09-4ef4-b0f4-933787c2e3e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:34.851868 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8b46b2c4-ce09-4ef4-b0f4-933787c2e3e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:34.943540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46b2c4-ce09-4ef4-b0f4-933787c2e3e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:34.945163 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b46b2c4-ce09-4ef4-b0f4-933787c2e3e6 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:34.963235 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 709/1419] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:36.105575 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a74c63-39bb-4c5a-94e3-4070ccb2fd18 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:36.106118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a74c63-39bb-4c5a-94e3-4070ccb2fd18 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:36.106118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a74c63-39bb-4c5a-94e3-4070ccb2fd18 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:36.107470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:56:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:36.113968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e68c3-6c8b-49e8-ab79-e3568861afc1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:36.114442 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e68c3-6c8b-49e8-ab79-e3568861afc1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:36.114538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255e68c3-6c8b-49e8-ab79-e3568861afc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:36.169112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:36.191991 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-255e68c3-6c8b-49e8-ab79-e3568861afc1 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:36.193233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-255e68c3-6c8b-49e8-ab79-e3568861afc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:36.290644 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:56:36.443567 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a74c63-39bb-4c5a-94e3-4070ccb2fd18 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:36.445122 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f9a74c63-39bb-4c5a-94e3-4070ccb2fd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:36.597307 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a74c63-39bb-4c5a-94e3-4070ccb2fd18 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:36.609631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:36.713576 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1dcfdc-d70f-4ae6-9e39-e39eb92f9826 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:36.714177 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1dcfdc-d70f-4ae6-9e39-e39eb92f9826 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:36.714453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1dcfdc-d70f-4ae6-9e39-e39eb92f9826 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:36.714719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1dcfdc-d70f-4ae6-9e39-e39eb92f9826 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:36.716323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3c1dcfdc-d70f-4ae6-9e39-e39eb92f9826 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:36.804272 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1dcfdc-d70f-4ae6-9e39-e39eb92f9826 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:36.805874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1dcfdc-d70f-4ae6-9e39-e39eb92f9826 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:36.824214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 712/1422] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:37.946771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56f0e0-218e-49fa-9893-81e85e3bf98b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:37.947374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56f0e0-218e-49fa-9893-81e85e3bf98b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:37.947374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56f0e0-218e-49fa-9893-81e85e3bf98b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:37.948745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:37.955179 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa925c-44df-4e8e-affa-ab0cd1046455 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:37.955594 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa925c-44df-4e8e-affa-ab0cd1046455 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:37.955766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa925c-44df-4e8e-affa-ab0cd1046455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:37.976905 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:38.036077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-55aa925c-44df-4e8e-affa-ab0cd1046455 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:38.037332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-55aa925c-44df-4e8e-affa-ab0cd1046455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:38.134097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:56:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:38.290800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56f0e0-218e-49fa-9893-81e85e3bf98b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:38.292263 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-da56f0e0-218e-49fa-9893-81e85e3bf98b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:38.403884 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:38.438920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56f0e0-218e-49fa-9893-81e85e3bf98b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:38.566791 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1327c16c-897a-418d-a1e6-346593559263 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:38.566791 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327c16c-897a-418d-a1e6-346593559263 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:38.566791 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327c16c-897a-418d-a1e6-346593559263 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:38.566791 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1327c16c-897a-418d-a1e6-346593559263 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:38.567399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1327c16c-897a-418d-a1e6-346593559263 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:38.638535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:38.651582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327c16c-897a-418d-a1e6-346593559263 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:38.653263 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1327c16c-897a-418d-a1e6-346593559263 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:38.671319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 712/1425] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:39.784694 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be223960-e82c-4026-91f1-37b6344c35a7 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:39.785277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be223960-e82c-4026-91f1-37b6344c35a7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:39.785475 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be223960-e82c-4026-91f1-37b6344c35a7 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:39.786826 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:56:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:39.793127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961d7fb-8f18-4034-ab85-f5953b6c6c16 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:39.793576 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961d7fb-8f18-4034-ab85-f5953b6c6c16 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:39.793683 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0961d7fb-8f18-4034-ab85-f5953b6c6c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:39.869531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0961d7fb-8f18-4034-ab85-f5953b6c6c16 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:39.870743 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0961d7fb-8f18-4034-ab85-f5953b6c6c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:39.968344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:56:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:40.135762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be223960-e82c-4026-91f1-37b6344c35a7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:40.137266 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-be223960-e82c-4026-91f1-37b6344c35a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:40.222098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:40.283888 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be223960-e82c-4026-91f1-37b6344c35a7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:40.395729 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74494c1-8d78-418c-9712-cde20388102d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:40.396249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74494c1-8d78-418c-9712-cde20388102d service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:40.396523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74494c1-8d78-418c-9712-cde20388102d service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:40.396790 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74494c1-8d78-418c-9712-cde20388102d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:40.398385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c74494c1-8d78-418c-9712-cde20388102d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:40.446532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:40.489361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74494c1-8d78-418c-9712-cde20388102d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:40.491386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74494c1-8d78-418c-9712-cde20388102d service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:40.509680 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 715/1428] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:41.628963 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c799cc-c7d5-4608-8bd5-9cc7e09f99db None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:41.629548 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c799cc-c7d5-4608-8bd5-9cc7e09f99db None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:41.629548 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c799cc-c7d5-4608-8bd5-9cc7e09f99db None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:41.630883 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:56:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:41.637278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f327b-01ff-4aab-8bf9-cc8f0cf4c557 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:41.637750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f327b-01ff-4aab-8bf9-cc8f0cf4c557 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:41.638197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819f327b-01ff-4aab-8bf9-cc8f0cf4c557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:41.714977 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-819f327b-01ff-4aab-8bf9-cc8f0cf4c557 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:41.716226 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-819f327b-01ff-4aab-8bf9-cc8f0cf4c557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:41.821138 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:56:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:41.978700 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c799cc-c7d5-4608-8bd5-9cc7e09f99db None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:41.980212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-05c799cc-c7d5-4608-8bd5-9cc7e09f99db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:42.030636 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:42.127830 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c799cc-c7d5-4608-8bd5-9cc7e09f99db None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:42.257617 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de60f85-21c7-4867-8494-c2fdd51233e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:42.258154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de60f85-21c7-4867-8494-c2fdd51233e3 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:42.258429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de60f85-21c7-4867-8494-c2fdd51233e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:42.258702 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de60f85-21c7-4867-8494-c2fdd51233e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:42.260521 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0de60f85-21c7-4867-8494-c2fdd51233e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:42.267679 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:42.351388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de60f85-21c7-4867-8494-c2fdd51233e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:42.353027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de60f85-21c7-4867-8494-c2fdd51233e3 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:42.370995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 715/1431] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:43.477787 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433466a4-57de-4ac5-986a-376503fa3dd9 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:43.478379 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433466a4-57de-4ac5-986a-376503fa3dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:43.478379 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433466a4-57de-4ac5-986a-376503fa3dd9 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:43.479765 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:56:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:43.486047 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62057e05-d7bf-40fc-a36c-09dbd81a7779 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:43.486469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62057e05-d7bf-40fc-a36c-09dbd81a7779 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:43.486580 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62057e05-d7bf-40fc-a36c-09dbd81a7779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:43.562738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-62057e05-d7bf-40fc-a36c-09dbd81a7779 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:43.563986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-62057e05-d7bf-40fc-a36c-09dbd81a7779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:43.634282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:43.663752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:43.815033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433466a4-57de-4ac5-986a-376503fa3dd9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:43.817607 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-433466a4-57de-4ac5-986a-376503fa3dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:43.835306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:43.970858 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433466a4-57de-4ac5-986a-376503fa3dd9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:44.061697 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:44.086676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd07756-cbda-45f3-bed8-0094fab62c87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:44.087392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd07756-cbda-45f3-bed8-0094fab62c87 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:44.087755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd07756-cbda-45f3-bed8-0094fab62c87 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:44.088229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd07756-cbda-45f3-bed8-0094fab62c87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:44.090221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6dd07756-cbda-45f3-bed8-0094fab62c87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:44.180942 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd07756-cbda-45f3-bed8-0094fab62c87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:44.182551 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd07756-cbda-45f3-bed8-0094fab62c87 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:44.200363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 718/1434] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:45.304955 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556cd79-ad6b-4661-8241-fb02f0e549ec None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:45.305477 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556cd79-ad6b-4661-8241-fb02f0e549ec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:45.305477 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556cd79-ad6b-4661-8241-fb02f0e549ec None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:45.306779 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:56:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:45.313267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cc77e-fd84-43d4-aae6-dc440247ace2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:45.313677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cc77e-fd84-43d4-aae6-dc440247ace2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:45.313848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cc77e-fd84-43d4-aae6-dc440247ace2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:45.389210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2f9cc77e-fd84-43d4-aae6-dc440247ace2 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:45.390427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2f9cc77e-fd84-43d4-aae6-dc440247ace2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:45.432556 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:45.488917 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:56:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:56:45.639712 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556cd79-ad6b-4661-8241-fb02f0e549ec None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:45.641276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2556cd79-ad6b-4661-8241-fb02f0e549ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:45.786115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556cd79-ad6b-4661-8241-fb02f0e549ec None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:45.841170 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:45.899859 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3004fe1-acc5-4386-98a4-e7f2edd02242 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:45.900365 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3004fe1-acc5-4386-98a4-e7f2edd02242 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:45.900643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3004fe1-acc5-4386-98a4-e7f2edd02242 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:45.900921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3004fe1-acc5-4386-98a4-e7f2edd02242 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:45.902498 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d3004fe1-acc5-4386-98a4-e7f2edd02242 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:45.996688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3004fe1-acc5-4386-98a4-e7f2edd02242 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:45.998273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3004fe1-acc5-4386-98a4-e7f2edd02242 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:46.017162 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 718/1437] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:47.165131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645970b7-4f5c-42a8-b1a0-7e1e99cf0708 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:47.165699 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645970b7-4f5c-42a8-b1a0-7e1e99cf0708 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:47.165699 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645970b7-4f5c-42a8-b1a0-7e1e99cf0708 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:47.167036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 17:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:47.173222 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6852af-c2f8-436f-87ba-69a29079f892 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:47.173679 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6852af-c2f8-436f-87ba-69a29079f892 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:47.173797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6852af-c2f8-436f-87ba-69a29079f892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:47.251989 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cf6852af-c2f8-436f-87ba-69a29079f892 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:47.255832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cf6852af-c2f8-436f-87ba-69a29079f892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:47.256120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:47.352031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 17:56:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:56:47.501574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645970b7-4f5c-42a8-b1a0-7e1e99cf0708 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:47.504025 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-645970b7-4f5c-42a8-b1a0-7e1e99cf0708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:47.650669 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645970b7-4f5c-42a8-b1a0-7e1e99cf0708 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:47.677441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:47.762678 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957f3a36-04f0-4ef7-9d44-895cf172e841 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:47.763464 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f3a36-04f0-4ef7-9d44-895cf172e841 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:47.763779 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f3a36-04f0-4ef7-9d44-895cf172e841 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:47.764054 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f3a36-04f0-4ef7-9d44-895cf172e841 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:47.765651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-957f3a36-04f0-4ef7-9d44-895cf172e841 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:47.854958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f3a36-04f0-4ef7-9d44-895cf172e841 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:47.856595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957f3a36-04f0-4ef7-9d44-895cf172e841 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:47.874791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 721/1440] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:49.952625 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d4fea-d95e-407b-9fb2-60bc9f371757 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:49.953171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d4fea-d95e-407b-9fb2-60bc9f371757 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:49.953171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d4fea-d95e-407b-9fb2-60bc9f371757 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:49.954536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:49.960097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07992f4a-28f8-47c2-9399-50b2fd6180af None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:49.960571 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07992f4a-28f8-47c2-9399-50b2fd6180af None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:49.960667 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07992f4a-28f8-47c2-9399-50b2fd6180af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:50.001618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:50.027325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-07992f4a-28f8-47c2-9399-50b2fd6180af None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:50.028565 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-07992f4a-28f8-47c2-9399-50b2fd6180af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:50.125077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:56:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:50.412280 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d4fea-d95e-407b-9fb2-60bc9f371757 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:50.413825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-213d4fea-d95e-407b-9fb2-60bc9f371757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:50.517075 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d4fea-d95e-407b-9fb2-60bc9f371757 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:50.551555 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:50.592946 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdad0a1-9659-4441-85bc-bff571c17b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:50.593542 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdad0a1-9659-4441-85bc-bff571c17b1c service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:50.593819 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdad0a1-9659-4441-85bc-bff571c17b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:50.594090 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdad0a1-9659-4441-85bc-bff571c17b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:50.595685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6fdad0a1-9659-4441-85bc-bff571c17b1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:50.702444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdad0a1-9659-4441-85bc-bff571c17b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:50.703831 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdad0a1-9659-4441-85bc-bff571c17b1c service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:50.722835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 721/1443] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:53.019262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d0c72-ad16-4feb-b854-3978cc12b745 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:53.019737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d0c72-ad16-4feb-b854-3978cc12b745 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:53.019841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1d0c72-ad16-4feb-b854-3978cc12b745 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:53.021172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:56:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:53.026297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f461e1-8b64-4933-a880-57455580993c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:53.026693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f461e1-8b64-4933-a880-57455580993c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:53.026843 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f461e1-8b64-4933-a880-57455580993c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:53.093242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c1f461e1-8b64-4933-a880-57455580993c None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:53.094466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c1f461e1-8b64-4933-a880-57455580993c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:53.108224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:53.195766 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:56:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:53.477051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d0c72-ad16-4feb-b854-3978cc12b745 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:53.478663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8d1d0c72-ad16-4feb-b854-3978cc12b745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:53.582923 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1d0c72-ad16-4feb-b854-3978cc12b745 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:53.595537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:53.655520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299fe474-cbff-42d0-88cb-c9b513e5be3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:53.656148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fe474-cbff-42d0-88cb-c9b513e5be3c service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:53.656427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fe474-cbff-42d0-88cb-c9b513e5be3c service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:53.656696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299fe474-cbff-42d0-88cb-c9b513e5be3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:53.658281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-299fe474-cbff-42d0-88cb-c9b513e5be3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:53.767976 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fe474-cbff-42d0-88cb-c9b513e5be3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:53.770016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299fe474-cbff-42d0-88cb-c9b513e5be3c service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:53.788466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 724/1446] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:56.324813 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ae2fc-e64e-4d1c-a629-3c220479f0b3 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:56.325330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ae2fc-e64e-4d1c-a629-3c220479f0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:56.325330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ae2fc-e64e-4d1c-a629-3c220479f0b3 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:56.326636 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:56:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:56.333070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795a125-3cae-446a-becd-8d302a8fcc73 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:56.333480 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795a125-3cae-446a-becd-8d302a8fcc73 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:56.333631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795a125-3cae-446a-becd-8d302a8fcc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:56.381555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:56.400363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b795a125-3cae-446a-becd-8d302a8fcc73 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:56.401569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b795a125-3cae-446a-becd-8d302a8fcc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:56.495740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:56:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:56.785682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ae2fc-e64e-4d1c-a629-3c220479f0b3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:56.787254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6b7ae2fc-e64e-4d1c-a629-3c220479f0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:56.891155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ae2fc-e64e-4d1c-a629-3c220479f0b3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:56:56.931312 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:56.966362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c58e32-fa8d-4127-ad33-1f7debae4c83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:56:56.966957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c58e32-fa8d-4127-ad33-1f7debae4c83 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:56.967234 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c58e32-fa8d-4127-ad33-1f7debae4c83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:56.967503 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c58e32-fa8d-4127-ad33-1f7debae4c83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:56.969105 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-66c58e32-fa8d-4127-ad33-1f7debae4c83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:57.079617 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c58e32-fa8d-4127-ad33-1f7debae4c83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:56:57.081757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c58e32-fa8d-4127-ad33-1f7debae4c83 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:56:57.099848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 724/1449] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:56:59.397926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a8c6e-28a5-464e-9f7d-30e19e8eb886 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:59.398483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a8c6e-28a5-464e-9f7d-30e19e8eb886 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:59.398483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87a8c6e-28a5-464e-9f7d-30e19e8eb886 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:59.399956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:56:59.405493 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627619e-b19a-4eed-8860-77757a332a18 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:56:59.405932 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627619e-b19a-4eed-8860-77757a332a18 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:56:59.406037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627619e-b19a-4eed-8860-77757a332a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:56:59.471515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b627619e-b19a-4eed-8860-77757a332a18 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:56:59.472752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b627619e-b19a-4eed-8860-77757a332a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:59.493787 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:56:59.569414 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:56:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:56:59.854145 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a8c6e-28a5-464e-9f7d-30e19e8eb886 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:56:59.855709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b87a8c6e-28a5-464e-9f7d-30e19e8eb886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:56:59.960037 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87a8c6e-28a5-464e-9f7d-30e19e8eb886 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:00.098797 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:00.152957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc91863-4a0d-4a46-bca7-a36c9e97fbec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:00.153554 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc91863-4a0d-4a46-bca7-a36c9e97fbec service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:00.153830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc91863-4a0d-4a46-bca7-a36c9e97fbec service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:00.154098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc91863-4a0d-4a46-bca7-a36c9e97fbec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:00.155691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-adc91863-4a0d-4a46-bca7-a36c9e97fbec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:00.265181 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc91863-4a0d-4a46-bca7-a36c9e97fbec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:00.266616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc91863-4a0d-4a46-bca7-a36c9e97fbec service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:00.284691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 727/1452] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:56:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:02.631204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93c1b8-6d00-4e81-96e8-e3fd8586d395 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:02.631808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93c1b8-6d00-4e81-96e8-e3fd8586d395 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:02.631808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93c1b8-6d00-4e81-96e8-e3fd8586d395 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:02.634636 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:57:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:02.640393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be20e9-88f1-4b00-b0c3-63adee148f11 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:02.640795 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be20e9-88f1-4b00-b0c3-63adee148f11 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:02.640975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98be20e9-88f1-4b00-b0c3-63adee148f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:02.703779 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:02.707305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-98be20e9-88f1-4b00-b0c3-63adee148f11 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:02.708534 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-98be20e9-88f1-4b00-b0c3-63adee148f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:02.804056 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:03.090389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93c1b8-6d00-4e81-96e8-e3fd8586d395 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:03.091995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ff93c1b8-6d00-4e81-96e8-e3fd8586d395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:03.199341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93c1b8-6d00-4e81-96e8-e3fd8586d395 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:03.249486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:03.274592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496956c6-8aaa-43a2-b0f3-209c7a012c00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:03.275200 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496956c6-8aaa-43a2-b0f3-209c7a012c00 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:03.275480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496956c6-8aaa-43a2-b0f3-209c7a012c00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:03.275808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496956c6-8aaa-43a2-b0f3-209c7a012c00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:03.277453 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-496956c6-8aaa-43a2-b0f3-209c7a012c00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:03.389032 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496956c6-8aaa-43a2-b0f3-209c7a012c00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:03.390470 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496956c6-8aaa-43a2-b0f3-209c7a012c00 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:03.408953 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 727/1455] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:05.762112 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54083f-0522-4630-94d1-eb73bb05ded2 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:05.762642 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54083f-0522-4630-94d1-eb73bb05ded2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:05.762642 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54083f-0522-4630-94d1-eb73bb05ded2 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:05.764207 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:05.769662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146628c9-670e-46bd-8972-5d8838023da5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:05.770103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146628c9-670e-46bd-8972-5d8838023da5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:05.770197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146628c9-670e-46bd-8972-5d8838023da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:05.834953 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-146628c9-670e-46bd-8972-5d8838023da5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:05.836201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-146628c9-670e-46bd-8972-5d8838023da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:05.865732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:05.932751 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:57:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:06.217334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54083f-0522-4630-94d1-eb73bb05ded2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:06.218881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bb54083f-0522-4630-94d1-eb73bb05ded2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:06.322495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54083f-0522-4630-94d1-eb73bb05ded2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:06.350731 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:06.397651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce6f039-bb06-4126-9bd0-27545cd7d1d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:06.398254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6f039-bb06-4126-9bd0-27545cd7d1d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:06.398530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6f039-bb06-4126-9bd0-27545cd7d1d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:06.398795 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce6f039-bb06-4126-9bd0-27545cd7d1d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:06.400400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1ce6f039-bb06-4126-9bd0-27545cd7d1d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:06.507869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6f039-bb06-4126-9bd0-27545cd7d1d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:06.509255 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce6f039-bb06-4126-9bd0-27545cd7d1d0 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:06.527816 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 730/1458] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:08.753864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad01f6d-4f4b-4460-9065-ffb97536d568 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:08.754439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad01f6d-4f4b-4460-9065-ffb97536d568 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:08.754439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad01f6d-4f4b-4460-9065-ffb97536d568 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:08.755818 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:57:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:08.761332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdf9d6-cb81-4074-ba47-0d8917aad326 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:08.761743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdf9d6-cb81-4074-ba47-0d8917aad326 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:08.761882 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bdf9d6-cb81-4074-ba47-0d8917aad326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:08.828219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-84bdf9d6-cb81-4074-ba47-0d8917aad326 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:08.829426 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-84bdf9d6-cb81-4074-ba47-0d8917aad326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:08.837825 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:08.928412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:57:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:57:09.221882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad01f6d-4f4b-4460-9065-ffb97536d568 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:09.223410 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3ad01f6d-4f4b-4460-9065-ffb97536d568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:09.327183 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad01f6d-4f4b-4460-9065-ffb97536d568 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:09.382113 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:09.400306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21394d5b-0210-485e-b543-9e5b1ab91df9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:09.400905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21394d5b-0210-485e-b543-9e5b1ab91df9 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:09.401178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21394d5b-0210-485e-b543-9e5b1ab91df9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:09.401444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21394d5b-0210-485e-b543-9e5b1ab91df9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:09.403015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-21394d5b-0210-485e-b543-9e5b1ab91df9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:09.513584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21394d5b-0210-485e-b543-9e5b1ab91df9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:09.515014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21394d5b-0210-485e-b543-9e5b1ab91df9 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:09.534074 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 730/1461] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:11.745709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069b395-6a08-4ede-874c-cdc89cdeea7b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:11.746313 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069b395-6a08-4ede-874c-cdc89cdeea7b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:11.746313 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069b395-6a08-4ede-874c-cdc89cdeea7b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:11.747579 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:11.753359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77ecb7-b7e5-4fea-b730-4d5e495f5b6b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:11.753773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77ecb7-b7e5-4fea-b730-4d5e495f5b6b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:11.753908 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77ecb7-b7e5-4fea-b730-4d5e495f5b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:11.820941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2e77ecb7-b7e5-4fea-b730-4d5e495f5b6b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:11.822152 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2e77ecb7-b7e5-4fea-b730-4d5e495f5b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:11.860007 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:11.918318 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:57:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:12.200437 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069b395-6a08-4ede-874c-cdc89cdeea7b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:12.202856 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6069b395-6a08-4ede-874c-cdc89cdeea7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:12.307370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069b395-6a08-4ede-874c-cdc89cdeea7b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:12.345200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:12.383522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4af8d-925f-45b8-9069-9c31009ea212 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:12.384163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4af8d-925f-45b8-9069-9c31009ea212 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:12.384473 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4af8d-925f-45b8-9069-9c31009ea212 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:12.384754 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4af8d-925f-45b8-9069-9c31009ea212 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:12.386345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-84e4af8d-925f-45b8-9069-9c31009ea212 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:12.507780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4af8d-925f-45b8-9069-9c31009ea212 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:12.509126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4af8d-925f-45b8-9069-9c31009ea212 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:12.527902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 733/1464] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:14.708196 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a898e-b6c5-43cc-9b85-aa3c431993d9 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:14.708759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a898e-b6c5-43cc-9b85-aa3c431993d9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:14.708759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a898e-b6c5-43cc-9b85-aa3c431993d9 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:14.710087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:14.715350 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bf336-9130-4f21-8b93-80ec884df8f4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:14.715822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bf336-9130-4f21-8b93-80ec884df8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:14.715930 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bf336-9130-4f21-8b93-80ec884df8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:14.782014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-590bf336-9130-4f21-8b93-80ec884df8f4 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:14.783769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-590bf336-9130-4f21-8b93-80ec884df8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:14.802316 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:14.880234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:57:15.171115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a898e-b6c5-43cc-9b85-aa3c431993d9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:15.172723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-705a898e-b6c5-43cc-9b85-aa3c431993d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:15.294579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a898e-b6c5-43cc-9b85-aa3c431993d9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:15.362453 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:15.369626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5604118-2b44-42a1-b2a8-56b9d5ad3781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:15.375247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5604118-2b44-42a1-b2a8-56b9d5ad3781 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:15.375247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5604118-2b44-42a1-b2a8-56b9d5ad3781 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:15.375247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5604118-2b44-42a1-b2a8-56b9d5ad3781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:15.375247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b5604118-2b44-42a1-b2a8-56b9d5ad3781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:15.607954 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5604118-2b44-42a1-b2a8-56b9d5ad3781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:15.609310 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5604118-2b44-42a1-b2a8-56b9d5ad3781 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:15.627832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 733/1467] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:17.838778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d961428-4b59-47db-9ede-432ca5af13b6 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:17.839320 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d961428-4b59-47db-9ede-432ca5af13b6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:17.839320 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d961428-4b59-47db-9ede-432ca5af13b6 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:17.840655 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 17:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:17.846573 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbf495-ab68-4205-b575-c2ca48017998 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:17.847015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbf495-ab68-4205-b575-c2ca48017998 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:17.847118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbf495-ab68-4205-b575-c2ca48017998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:17.912085 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0adbf495-ab68-4205-b575-c2ca48017998 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:17.913279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0adbf495-ab68-4205-b575-c2ca48017998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:17.960960 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:18.014184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 17:57:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:18.300054 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d961428-4b59-47db-9ede-432ca5af13b6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:18.301588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9d961428-4b59-47db-9ede-432ca5af13b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:18.403926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d961428-4b59-47db-9ede-432ca5af13b6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:18.452803 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:18.477203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7235b14b-eac4-48a4-af94-9a5a38a4c1d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:18.477799 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235b14b-eac4-48a4-af94-9a5a38a4c1d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:18.478080 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235b14b-eac4-48a4-af94-9a5a38a4c1d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:18.478348 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7235b14b-eac4-48a4-af94-9a5a38a4c1d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:18.479977 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7235b14b-eac4-48a4-af94-9a5a38a4c1d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:18.596893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235b14b-eac4-48a4-af94-9a5a38a4c1d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:18.598238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7235b14b-eac4-48a4-af94-9a5a38a4c1d4 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:18.616467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 736/1470] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 17:57:28.011808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=22 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:28.614212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=66 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:39.031786 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb15038-29f4-4d13-8585-ad0b4c467c2a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:39.032327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb15038-29f4-4d13-8585-ad0b4c467c2a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:39.032448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb15038-29f4-4d13-8585-ad0b4c467c2a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:39.033720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 735/1471] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:39.039267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcff597-f539-467a-b6b5-3294dff55cc5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:39.039892 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcff597-f539-467a-b6b5-3294dff55cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:39.039892 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcff597-f539-467a-b6b5-3294dff55cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:39.105679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7dcff597-f539-467a-b6b5-3294dff55cc5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:39.106894 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7dcff597-f539-467a-b6b5-3294dff55cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:39.204127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 737/1472] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:39.506325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f89a5-3b6f-4fcc-83d1-d66738f49e51 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:39.506610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f89a5-3b6f-4fcc-83d1-d66738f49e51 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:39.506913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f89a5-3b6f-4fcc-83d1-d66738f49e51 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:39.508286 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 736/1473] 10.222.0.67 () {54 vars in 962 bytes} [Mon Sep 26 17:57:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:39.513386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0b523-6ea9-4f25-b7e7-978d89847c7c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:39.513670 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0b523-6ea9-4f25-b7e7-978d89847c7c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:39.513941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0b523-6ea9-4f25-b7e7-978d89847c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:39.591146 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-05c0b523-6ea9-4f25-b7e7-978d89847c7c None None] MFA Rules not processed for user `e6dee005fc3d4fb593a1771e7851c20a`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:39.592386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-05c0b523-6ea9-4f25-b7e7-978d89847c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:39.614277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:39.670762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 738/1474] 10.222.0.67 () {58 vars in 1059 bytes} [Mon Sep 26 17:57:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:39.679048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f89a5-3b6f-4fcc-83d1-d66738f49e51 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:39.680573 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-670f89a5-3b6f-4fcc-83d1-d66738f49e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:39.784563 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f89a5-3b6f-4fcc-83d1-d66738f49e51 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:39.856813 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36334307-73d5-44c6-abae-18c2262e7011 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6dee005fc3d4fb593a1771e7851c20a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:39.857408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36334307-73d5-44c6-abae-18c2262e7011 service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:39.857686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36334307-73d5-44c6-abae-18c2262e7011 service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:39.857954 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36334307-73d5-44c6-abae-18c2262e7011 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:39.859536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-36334307-73d5-44c6-abae-18c2262e7011 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:39.915070 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:39.970220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36334307-73d5-44c6-abae-18c2262e7011 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:39.971678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36334307-73d5-44c6-abae-18c2262e7011 service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:39.991883 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 737/1475] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:42.270960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d47510-1416-46e7-872b-9cb0ad98bd0f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:42.271525 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d47510-1416-46e7-872b-9cb0ad98bd0f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:42.271525 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d47510-1416-46e7-872b-9cb0ad98bd0f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:42.272781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 739/1476] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:57:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:42.278118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad69e92-9545-463a-84a7-ea17d084df80 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:42.278571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad69e92-9545-463a-84a7-ea17d084df80 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:42.278677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad69e92-9545-463a-84a7-ea17d084df80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:42.343797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1ad69e92-9545-463a-84a7-ea17d084df80 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:42.345129 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1ad69e92-9545-463a-84a7-ea17d084df80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:42.440672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 738/1477] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:57:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:42.713387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d47510-1416-46e7-872b-9cb0ad98bd0f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:42.714916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-80d47510-1416-46e7-872b-9cb0ad98bd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:42.838804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d47510-1416-46e7-872b-9cb0ad98bd0f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:42.888392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:42.933327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d43de5-1444-4055-ae3c-c9e2ef00a669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:42.933960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d43de5-1444-4055-ae3c-c9e2ef00a669 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:42.934239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d43de5-1444-4055-ae3c-c9e2ef00a669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:42.934501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d43de5-1444-4055-ae3c-c9e2ef00a669 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:42.935123 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d43de5-1444-4055-ae3c-c9e2ef00a669 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:42.936878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-e4d43de5-1444-4055-ae3c-c9e2ef00a669 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:57:42.936878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 26 17:57:42.936878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: Sep 26 17:57:42.938173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d43de5-1444-4055-ae3c-c9e2ef00a669 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:42.958466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 740/1478] 10.222.0.67 () {56 vars in 1108 bytes} [Mon Sep 26 17:57:42 2022] GET /identity/v3/projects => generated 2553 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:57:45.152547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069d5ee-eeb5-4ac0-a724-b0f7d73479d7 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:45.153108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069d5ee-eeb5-4ac0-a724-b0f7d73479d7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:45.153108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069d5ee-eeb5-4ac0-a724-b0f7d73479d7 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:45.154441 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 739/1479] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:45.159672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba7a8f-1248-4e22-907a-9568f23e0c5a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:45.160103 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba7a8f-1248-4e22-907a-9568f23e0c5a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:45.160311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ba7a8f-1248-4e22-907a-9568f23e0c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:45.226298 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e3ba7a8f-1248-4e22-907a-9568f23e0c5a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:45.227505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e3ba7a8f-1248-4e22-907a-9568f23e0c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:45.324975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 741/1480] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:57:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:45.619433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069d5ee-eeb5-4ac0-a724-b0f7d73479d7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:45.620995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3069d5ee-eeb5-4ac0-a724-b0f7d73479d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:45.630768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:45.766036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069d5ee-eeb5-4ac0-a724-b0f7d73479d7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:45.843797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:45.876573 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9967f5e2-dd0d-4647-b576-1ec4300a13cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:45.877173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967f5e2-dd0d-4647-b576-1ec4300a13cd service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:45.877450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967f5e2-dd0d-4647-b576-1ec4300a13cd service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:45.877719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9967f5e2-dd0d-4647-b576-1ec4300a13cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:45.879287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9967f5e2-dd0d-4647-b576-1ec4300a13cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:45.968018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967f5e2-dd0d-4647-b576-1ec4300a13cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:45.970183 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9967f5e2-dd0d-4647-b576-1ec4300a13cd service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:45.990530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 740/1481] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:48.387020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57e6ff-ff74-437e-b04b-b64d9e6b0c77 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:48.387830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57e6ff-ff74-437e-b04b-b64d9e6b0c77 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:48.387830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57e6ff-ff74-437e-b04b-b64d9e6b0c77 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:48.389186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 742/1482] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:57:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:48.394597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082dffd7-3228-4fd9-b777-132a1043d97f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:48.395017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082dffd7-3228-4fd9-b777-132a1043d97f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:48.395129 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082dffd7-3228-4fd9-b777-132a1043d97f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:48.460205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-082dffd7-3228-4fd9-b777-132a1043d97f None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:48.461415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-082dffd7-3228-4fd9-b777-132a1043d97f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:48.508034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:48.557758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 741/1483] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:57:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:48.844517 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57e6ff-ff74-437e-b04b-b64d9e6b0c77 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:48.846058 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0a57e6ff-ff74-437e-b04b-b64d9e6b0c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:48.870326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:48.955132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57e6ff-ff74-437e-b04b-b64d9e6b0c77 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:49.031105 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e5899-fa2b-427e-941f-07b26715920e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:49.032390 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e5899-fa2b-427e-941f-07b26715920e service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:49.032669 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e5899-fa2b-427e-941f-07b26715920e service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:49.032939 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e5899-fa2b-427e-941f-07b26715920e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:49.034531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e72e5899-fa2b-427e-941f-07b26715920e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:49.109099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:49.145005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e5899-fa2b-427e-941f-07b26715920e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:49.146458 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e5899-fa2b-427e-941f-07b26715920e service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:49.165115 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 743/1484] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:51.371217 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145046bf-b7f9-4b15-818e-29ff79b366ef None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:51.371800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145046bf-b7f9-4b15-818e-29ff79b366ef None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:51.371800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145046bf-b7f9-4b15-818e-29ff79b366ef None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:51.373173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 742/1485] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:57:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:51.378870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8493d-a10c-4066-81c9-07ef1c8d7726 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:51.379327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8493d-a10c-4066-81c9-07ef1c8d7726 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:51.379457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8493d-a10c-4066-81c9-07ef1c8d7726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:51.444593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e4f8493d-a10c-4066-81c9-07ef1c8d7726 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:51.445806 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e4f8493d-a10c-4066-81c9-07ef1c8d7726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:51.544615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 744/1486] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:57:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:51.842475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145046bf-b7f9-4b15-818e-29ff79b366ef None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:51.844771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-145046bf-b7f9-4b15-818e-29ff79b366ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:51.947878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145046bf-b7f9-4b15-818e-29ff79b366ef None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:52.022120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:52.023673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6169686-607b-4bbe-acd0-5d297015d018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:52.024279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6169686-607b-4bbe-acd0-5d297015d018 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:52.024555 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6169686-607b-4bbe-acd0-5d297015d018 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:52.024825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6169686-607b-4bbe-acd0-5d297015d018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:52.026408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d6169686-607b-4bbe-acd0-5d297015d018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:52.134568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6169686-607b-4bbe-acd0-5d297015d018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:52.136032 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6169686-607b-4bbe-acd0-5d297015d018 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:52.154462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 743/1487] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:54.446947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d5a91-f44c-46c9-a7ae-d9d37172af9c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:54.447487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d5a91-f44c-46c9-a7ae-d9d37172af9c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:54.447487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d5a91-f44c-46c9-a7ae-d9d37172af9c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:54.448845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 745/1488] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:54.455094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d0f83-2e6e-4531-b215-d79ed89cf1a5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:54.455535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d0f83-2e6e-4531-b215-d79ed89cf1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:54.455721 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d0f83-2e6e-4531-b215-d79ed89cf1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:54.521755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-084d0f83-2e6e-4531-b215-d79ed89cf1a5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:54.522956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-084d0f83-2e6e-4531-b215-d79ed89cf1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:54.576907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:54.622590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 744/1489] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:57:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:54.913711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d5a91-f44c-46c9-a7ae-d9d37172af9c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:54.915240 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cb8d5a91-f44c-46c9-a7ae-d9d37172af9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:54.951255 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:55.021557 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d5a91-f44c-46c9-a7ae-d9d37172af9c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:55.096536 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c8d9a3-4a4c-47d5-a7aa-77ac67f8c5ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:55.097143 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8d9a3-4a4c-47d5-a7aa-77ac67f8c5ee service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:55.097425 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8d9a3-4a4c-47d5-a7aa-77ac67f8c5ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:55.097697 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c8d9a3-4a4c-47d5-a7aa-77ac67f8c5ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:55.099290 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-98c8d9a3-4a4c-47d5-a7aa-77ac67f8c5ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:55.180419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:55.209783 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c8d9a3-4a4c-47d5-a7aa-77ac67f8c5ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:55.211469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c8d9a3-4a4c-47d5-a7aa-77ac67f8c5ee service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:55.229349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 746/1490] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:57:57.741369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae6fa0-ab8a-41b5-aa33-8b45609a3c42 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:57.741976 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae6fa0-ab8a-41b5-aa33-8b45609a3c42 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:57.741976 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ae6fa0-ab8a-41b5-aa33-8b45609a3c42 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:57.743263 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 745/1491] 10.222.0.67 () {54 vars in 927 bytes} [Mon Sep 26 17:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:57:57.748702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9fe2c-03e8-4fa5-aec9-138ea8286d4e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:57.749114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9fe2c-03e8-4fa5-aec9-138ea8286d4e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:57.749237 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9fe2c-03e8-4fa5-aec9-138ea8286d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:57.818017 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-24b9fe2c-03e8-4fa5-aec9-138ea8286d4e None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:57:57.819143 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-24b9fe2c-03e8-4fa5-aec9-138ea8286d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:57.916542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 747/1492] 10.222.0.67 () {58 vars in 1024 bytes} [Mon Sep 26 17:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:57:58.205709 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae6fa0-ab8a-41b5-aa33-8b45609a3c42 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:57:58.207278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c6ae6fa0-ab8a-41b5-aa33-8b45609a3c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:58.312081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ae6fa0-ab8a-41b5-aa33-8b45609a3c42 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:57:58.385814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ddeba-a4ad-40c2-9528-4699ab557a1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:57:58.386417 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ddeba-a4ad-40c2-9528-4699ab557a1b service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:57:58.386695 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ddeba-a4ad-40c2-9528-4699ab557a1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:57:58.386966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ddeba-a4ad-40c2-9528-4699ab557a1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:57:58.388867 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d14ddeba-a4ad-40c2-9528-4699ab557a1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:57:58.397720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:57:58.500698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ddeba-a4ad-40c2-9528-4699ab557a1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:57:58.502048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ddeba-a4ad-40c2-9528-4699ab557a1b service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:57:58.520901 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 746/1493] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:57:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:58:07.914977 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=82 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:58:08.518571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=50 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:22.698999 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164b782-68d5-40b3-ba3a-789bfa7be6a0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:22.699603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164b782-68d5-40b3-ba3a-789bfa7be6a0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:22.699603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164b782-68d5-40b3-ba3a-789bfa7be6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:22.785555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7164b782-68d5-40b3-ba3a-789bfa7be6a0 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:22.786788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7164b782-68d5-40b3-ba3a-789bfa7be6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:22.890075 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 748/1494] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 17:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 17:59:22.898188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13addc87-271a-4b11-899e-6ac1e284f3fb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:22.898713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13addc87-271a-4b11-899e-6ac1e284f3fb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:22.898713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13addc87-271a-4b11-899e-6ac1e284f3fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:22.978667 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-13addc87-271a-4b11-899e-6ac1e284f3fb None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:22.979857 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-13addc87-271a-4b11-899e-6ac1e284f3fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:23.082595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 747/1495] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 17:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:23.087352 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164b782-68d5-40b3-ba3a-789bfa7be6a0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:23.088613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7164b782-68d5-40b3-ba3a-789bfa7be6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:23.100555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164b782-68d5-40b3-ba3a-789bfa7be6a0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:23.179771 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:23.218303 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db580a8-d31b-4e05-b544-548fdc59fa97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:23.218920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db580a8-d31b-4e05-b544-548fdc59fa97 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:23.219199 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db580a8-d31b-4e05-b544-548fdc59fa97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:23.219476 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db580a8-d31b-4e05-b544-548fdc59fa97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:23.220161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db580a8-d31b-4e05-b544-548fdc59fa97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 17:59:23.220467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db580a8-d31b-4e05-b544-548fdc59fa97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:23.221492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-9db580a8-d31b-4e05-b544-548fdc59fa97 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:59:23.221492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 26 17:59:23.221492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: Sep 26 17:59:23.222369 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db580a8-d31b-4e05-b544-548fdc59fa97 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:23.235345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 749/1496] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 17:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 17:59:23.243704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13addc87-271a-4b11-899e-6ac1e284f3fb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:23.254536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13addc87-271a-4b11-899e-6ac1e284f3fb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:23.341523 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:23.386775 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d4c73-e345-4e7a-bb1a-3816b669abdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:23.387676 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d4c73-e345-4e7a-bb1a-3816b669abdf admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:23.387954 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d4c73-e345-4e7a-bb1a-3816b669abdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:23.388392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d4c73-e345-4e7a-bb1a-3816b669abdf admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:23.389097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d4c73-e345-4e7a-bb1a-3816b669abdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:23.390921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-a53d4c73-e345-4e7a-bb1a-3816b669abdf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:59:23.390921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Sep 26 17:59:23.390921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: Sep 26 17:59:23.392154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d4c73-e345-4e7a-bb1a-3816b669abdf admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:23.578419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 748/1497] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 17:59:23 2022] POST /identity/v3/projects => generated 379 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:23.583876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db580a8-d31b-4e05-b544-548fdc59fa97 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:23.585523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9db580a8-d31b-4e05-b544-548fdc59fa97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:23.624254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db580a8-d31b-4e05-b544-548fdc59fa97 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:23.725028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e4c1e-415c-4883-8081-b4d96bf71bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:23.725723 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e4c1e-415c-4883-8081-b4d96bf71bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:23.725899 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e4c1e-415c-4883-8081-b4d96bf71bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:23.726166 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692e4c1e-415c-4883-8081-b4d96bf71bf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:23.726789 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e4c1e-415c-4883-8081-b4d96bf71bf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:23.727873 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-692e4c1e-415c-4883-8081-b4d96bf71bf4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:59:23.727873 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 26 17:59:23.727873 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: Sep 26 17:59:23.730079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692e4c1e-415c-4883-8081-b4d96bf71bf4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:23.834745 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 750/1498] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 17:59:23 2022] POST /identity/v3/users => generated 290 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 17:59:23.839042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d4c73-e345-4e7a-bb1a-3816b669abdf admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:23.841090 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a53d4c73-e345-4e7a-bb1a-3816b669abdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:23.906354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d4c73-e345-4e7a-bb1a-3816b669abdf admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:23.998175 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:24.008474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a17c4a-e4f5-464a-9a1a-703afdcffa25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:24.011411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a17c4a-e4f5-464a-9a1a-703afdcffa25 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:24.011411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a17c4a-e4f5-464a-9a1a-703afdcffa25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:24.011411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a17c4a-e4f5-464a-9a1a-703afdcffa25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:24.011411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a17c4a-e4f5-464a-9a1a-703afdcffa25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:24.011411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-25a17c4a-e4f5-464a-9a1a-703afdcffa25 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:59:24.011411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Sep 26 17:59:24.011411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: Sep 26 17:59:24.012673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a17c4a-e4f5-464a-9a1a-703afdcffa25 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:24.026495 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 749/1499] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:23 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:24.031224 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e4c1e-415c-4883-8081-b4d96bf71bf4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:24.032789 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-692e4c1e-415c-4883-8081-b4d96bf71bf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:24.043971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692e4c1e-415c-4883-8081-b4d96bf71bf4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:24.068673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:24.144598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9ac145-f214-46e6-9ef2-63cfb261282d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:24.145204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ac145-f214-46e6-9ef2-63cfb261282d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:24.145476 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ac145-f214-46e6-9ef2-63cfb261282d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:24.145748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9ac145-f214-46e6-9ef2-63cfb261282d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:24.146379 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ac145-f214-46e6-9ef2-63cfb261282d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:24.147389 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-8c9ac145-f214-46e6-9ef2-63cfb261282d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:59:24.147389 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 26 17:59:24.147389 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: Sep 26 17:59:24.148293 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9ac145-f214-46e6-9ef2-63cfb261282d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:24.161169 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 751/1500] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 17:59:24 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 17:59:24.165409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a17c4a-e4f5-464a-9a1a-703afdcffa25 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:24.166941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-25a17c4a-e4f5-464a-9a1a-703afdcffa25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:24.201337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a17c4a-e4f5-464a-9a1a-703afdcffa25 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:24.295465 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07a8d4-fe2a-40fc-b435-dbbaadbdced6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:24.296146 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07a8d4-fe2a-40fc-b435-dbbaadbdced6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:24.296425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07a8d4-fe2a-40fc-b435-dbbaadbdced6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:24.296697 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07a8d4-fe2a-40fc-b435-dbbaadbdced6 admin admin] PATH_INFO: `/v3/projects/51a6803ac07e4f5eafb7d17154fd46da/users/bb65531bf50444f891f982e53a2cfd71/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:24.319629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de07a8d4-fe2a-40fc-b435-dbbaadbdced6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51a6803ac07e4f5eafb7d17154fd46da, user_id=bb65531bf50444f891f982e53a2cfd71, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:24.320724 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-de07a8d4-fe2a-40fc-b435-dbbaadbdced6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:59:24.320724 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Sep 26 17:59:24.320724 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: Sep 26 17:59:24.322650 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de07a8d4-fe2a-40fc-b435-dbbaadbdced6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:24.411252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 750/1501] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 17:59:24 2022] PUT /identity/v3/projects/51a6803ac07e4f5eafb7d17154fd46da/users/bb65531bf50444f891f982e53a2cfd71/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:24.424460 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a0558-f14e-4085-918b-4a9693e18a20 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:24.424747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a0558-f14e-4085-918b-4a9693e18a20 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:24.425028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a0558-f14e-4085-918b-4a9693e18a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:24.482842 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ec4a0558-f14e-4085-918b-4a9693e18a20 None None] MFA Rules not processed for user `bb65531bf50444f891f982e53a2cfd71`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:24.484104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ec4a0558-f14e-4085-918b-4a9693e18a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:24.562021 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 752/1502] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 17:59:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:24.575143 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935389b-7b2f-4a9c-8f35-72a8dc7ae761 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:24.575431 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935389b-7b2f-4a9c-8f35-72a8dc7ae761 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:24.576308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d935389b-7b2f-4a9c-8f35-72a8dc7ae761 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:24.577842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 751/1503] 10.222.0.67 () {54 vars in 963 bytes} [Mon Sep 26 17:59:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:59:24.582874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7160d29-7193-4bf7-819b-3e2feb2d8ce7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:24.583164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7160d29-7193-4bf7-819b-3e2feb2d8ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:24.583441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7160d29-7193-4bf7-819b-3e2feb2d8ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:24.616257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:24.662981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e7160d29-7193-4bf7-819b-3e2feb2d8ce7 None None] MFA Rules not processed for user `e6d98d51966f48d3a1b22f9b1c7963be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:24.664148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e7160d29-7193-4bf7-819b-3e2feb2d8ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:24.742041 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 753/1504] 10.222.0.67 () {58 vars in 1060 bytes} [Mon Sep 26 17:59:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:24.750622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935389b-7b2f-4a9c-8f35-72a8dc7ae761 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:24.752161 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d935389b-7b2f-4a9c-8f35-72a8dc7ae761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:24.801325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:24.853899 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d935389b-7b2f-4a9c-8f35-72a8dc7ae761 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:24.942064 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b03f4-3504-428b-b67c-5bb9c1cfbea1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:24.942715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b03f4-3504-428b-b67c-5bb9c1cfbea1 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:24.942853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b03f4-3504-428b-b67c-5bb9c1cfbea1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:24.943121 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b03f4-3504-428b-b67c-5bb9c1cfbea1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:24.944768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c04b03f4-3504-428b-b67c-5bb9c1cfbea1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:25.056839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b03f4-3504-428b-b67c-5bb9c1cfbea1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:25.059030 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04b03f4-3504-428b-b67c-5bb9c1cfbea1 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:25.063892 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:25.084401 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 752/1505] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:24 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:25.131456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34734640-70ce-408b-9279-959013731e4b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:25.131783 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34734640-70ce-408b-9279-959013731e4b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:25.132059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34734640-70ce-408b-9279-959013731e4b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:25.133332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 754/1506] 10.222.0.67 () {52 vars in 830 bytes} [Mon Sep 26 17:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 17:59:25.139079 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b03f4-3504-428b-b67c-5bb9c1cfbea1 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:25.140546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c04b03f4-3504-428b-b67c-5bb9c1cfbea1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:25.241507 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04b03f4-3504-428b-b67c-5bb9c1cfbea1 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:25.336660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63180391-c036-443b-ade9-8ab6ab2f3187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:25.337288 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63180391-c036-443b-ade9-8ab6ab2f3187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:25.337564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63180391-c036-443b-ade9-8ab6ab2f3187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:25.337832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63180391-c036-443b-ade9-8ab6ab2f3187 admin admin] PATH_INFO: `/v3/users/bb65531bf50444f891f982e53a2cfd71` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:25.351004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63180391-c036-443b-ade9-8ab6ab2f3187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb65531bf50444f891f982e53a2cfd71)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:25.352086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-63180391-c036-443b-ade9-8ab6ab2f3187 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:59:25.352086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Sep 26 17:59:25.352086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: Sep 26 17:59:25.353295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63180391-c036-443b-ade9-8ab6ab2f3187 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:25.363853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:25.577510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-63180391-c036-443b-ade9-8ab6ab2f3187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb65531bf50444f891f982e53a2cfd71'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:25.650114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-63180391-c036-443b-ade9-8ab6ab2f3187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb65531bf50444f891f982e53a2cfd71'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:25.650655 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-63180391-c036-443b-ade9-8ab6ab2f3187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb65531bf50444f891f982e53a2cfd71'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:25.651122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-63180391-c036-443b-ade9-8ab6ab2f3187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb65531bf50444f891f982e53a2cfd71'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:25.669485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-63180391-c036-443b-ade9-8ab6ab2f3187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb65531bf50444f891f982e53a2cfd71'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:25.701437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 753/1507] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 17:59:25 2022] DELETE /identity/v3/users/bb65531bf50444f891f982e53a2cfd71 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:25.705915 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683010e-32de-40f1-815f-815fe691dd06 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:25.706197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683010e-32de-40f1-815f-815fe691dd06 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:25.706467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2683010e-32de-40f1-815f-815fe691dd06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:25.776016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2683010e-32de-40f1-815f-815fe691dd06 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 17:59:25.777239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2683010e-32de-40f1-815f-815fe691dd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:25.860395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:25.871725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 755/1508] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 17:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 17:59:25.882391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63180391-c036-443b-ade9-8ab6ab2f3187 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:25.883621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-63180391-c036-443b-ade9-8ab6ab2f3187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:25.993291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63180391-c036-443b-ade9-8ab6ab2f3187 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:26.059408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:26.094308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1c8d7-f374-41fa-b551-277ed9d86029 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:26.095337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1c8d7-f374-41fa-b551-277ed9d86029 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:26.095808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1c8d7-f374-41fa-b551-277ed9d86029 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:26.096200 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1c8d7-f374-41fa-b551-277ed9d86029 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:26.098943 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e9f1c8d7-f374-41fa-b551-277ed9d86029 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:26.222145 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1c8d7-f374-41fa-b551-277ed9d86029 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:26.223602 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1c8d7-f374-41fa-b551-277ed9d86029 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:26.242575 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 754/1509] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 17:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 17:59:26.917998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2683010e-32de-40f1-815f-815fe691dd06 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 17:59:26.924377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2683010e-32de-40f1-815f-815fe691dd06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 17:59:26.930906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2683010e-32de-40f1-815f-815fe691dd06 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 17:59:27.047016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 17:59:27.047732 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 17:59:27.047973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 17:59:27.048249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924 admin admin] PATH_INFO: `/v3/projects/51a6803ac07e4f5eafb7d17154fd46da` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 17:59:27.058813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51a6803ac07e4f5eafb7d17154fd46da)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 17:59:27.059953 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 17:59:27.059953 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 26 17:59:27.059953 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: Sep 26 17:59:27.061617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 17:59:27.184458 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51a6803ac07e4f5eafb7d17154fd46da', 'request_id': 'req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:27.199867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51a6803ac07e4f5eafb7d17154fd46da', 'request_id': 'req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:27.269952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51a6803ac07e4f5eafb7d17154fd46da', 'request_id': 'req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:27.295272 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924 admin admin] The token cache is being invalidate because project 51a6803ac07e4f5eafb7d17154fd46da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 17:59:27.295570 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:27.296280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c29cadb-3b43-4ebd-9d7c-0443e75d4924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 17:59:27.297624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 756/1510] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 17:59:26 2022] DELETE /identity/v3/projects/51a6803ac07e4f5eafb7d17154fd46da => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 17:59:36.240329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=62 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 17:59:37.292972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:05.259459 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bc4e9-d623-4c89-a953-5c6fec5b0d33 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:05.260077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bc4e9-d623-4c89-a953-5c6fec5b0d33 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:05.260077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940bc4e9-d623-4c89-a953-5c6fec5b0d33 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:05.261526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 18:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:01:05.267160 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e1c46-6339-403b-95cd-33f9c91be6f5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:05.267608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e1c46-6339-403b-95cd-33f9c91be6f5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:05.267799 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7e1c46-6339-403b-95cd-33f9c91be6f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:05.352549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-de7e1c46-6339-403b-95cd-33f9c91be6f5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:05.353782 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-de7e1c46-6339-403b-95cd-33f9c91be6f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:05.470813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 18:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:05.626089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bc4e9-d623-4c89-a953-5c6fec5b0d33 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:05.628189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-940bc4e9-d623-4c89-a953-5c6fec5b0d33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:05.752033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940bc4e9-d623-4c89-a953-5c6fec5b0d33 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:05.759280 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a46c8-e0c9-4aaf-9ae1-e3faf64c3d77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:05.759936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a46c8-e0c9-4aaf-9ae1-e3faf64c3d77 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:05.760213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a46c8-e0c9-4aaf-9ae1-e3faf64c3d77 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:05.760478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a46c8-e0c9-4aaf-9ae1-e3faf64c3d77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:05.770417 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a46c8-e0c9-4aaf-9ae1-e3faf64c3d77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:05.772162 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a46c8-e0c9-4aaf-9ae1-e3faf64c3d77 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:05.783485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 756/1513] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:07.004591 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97df31e-b759-44a6-b0c3-819207d22c26 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:07.005060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97df31e-b759-44a6-b0c3-819207d22c26 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:07.005173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97df31e-b759-44a6-b0c3-819207d22c26 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:07.006500 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 18:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:01:07.013150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5885919-f292-4d19-b064-6052f333a6cc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:07.014230 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5885919-f292-4d19-b064-6052f333a6cc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:07.014320 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5885919-f292-4d19-b064-6052f333a6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:07.034733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a5885919-f292-4d19-b064-6052f333a6cc None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:07.036002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a5885919-f292-4d19-b064-6052f333a6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:07.043691 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 18:01:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:07.200148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97df31e-b759-44a6-b0c3-819207d22c26 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:07.215878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97df31e-b759-44a6-b0c3-819207d22c26 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:07.222096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb24f6d8-43e0-49c7-a252-4f5940e0b996 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:07.222706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24f6d8-43e0-49c7-a252-4f5940e0b996 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:07.222984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24f6d8-43e0-49c7-a252-4f5940e0b996 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:07.223257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb24f6d8-43e0-49c7-a252-4f5940e0b996 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:07.231808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24f6d8-43e0-49c7-a252-4f5940e0b996 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:07.234663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb24f6d8-43e0-49c7-a252-4f5940e0b996 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:07.244941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 759/1516] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:08.616768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bf316-2e37-4568-9771-654300f08e37 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:08.617341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bf316-2e37-4568-9771-654300f08e37 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:08.617341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025bf316-2e37-4568-9771-654300f08e37 None None] PATH_INFO: `/v3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:08.618616 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 758/1517] 10.222.0.88 () {54 vars in 925 bytes} [Mon Sep 26 18:01:08 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:01:08.625136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b13bfe-d0a1-47e6-828f-aeb444b5543e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:08.625613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b13bfe-d0a1-47e6-828f-aeb444b5543e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:08.625710 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b13bfe-d0a1-47e6-828f-aeb444b5543e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:08.687345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-11b13bfe-d0a1-47e6-828f-aeb444b5543e None None] MFA Rules not processed for user `22a512e0a64e420abf15b2150ccc2b32`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:08.688611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-11b13bfe-d0a1-47e6-828f-aeb444b5543e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:08.705947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:08.764815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 760/1518] 10.222.0.88 () {58 vars in 1015 bytes} [Mon Sep 26 18:01:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:17.029449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=56 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:18.760644 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=23 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:18.768951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3287ee-6061-4dd8-82ed-9bb5f95760c1 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:18.776883 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3287ee-6061-4dd8-82ed-9bb5f95760c1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:18.776883 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3287ee-6061-4dd8-82ed-9bb5f95760c1 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:18.776883 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 759/1519] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 18:01:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:01:18.777645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f08a4b-4599-4d6a-a21b-d65541180e1b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:18.777936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f08a4b-4599-4d6a-a21b-d65541180e1b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:18.778210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f08a4b-4599-4d6a-a21b-d65541180e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:18.802613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-05f08a4b-4599-4d6a-a21b-d65541180e1b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:18.807961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-05f08a4b-4599-4d6a-a21b-d65541180e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:18.812742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 761/1520] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 18:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:18.969118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3287ee-6061-4dd8-82ed-9bb5f95760c1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:18.986440 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3287ee-6061-4dd8-82ed-9bb5f95760c1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:18.992086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f02b9-6c75-4d38-be22-100eca61c645 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:18.992700 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f02b9-6c75-4d38-be22-100eca61c645 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:18.992978 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f02b9-6c75-4d38-be22-100eca61c645 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:18.993250 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f02b9-6c75-4d38-be22-100eca61c645 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:19.002921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f02b9-6c75-4d38-be22-100eca61c645 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:19.004562 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7f02b9-6c75-4d38-be22-100eca61c645 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:19.015038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 760/1521] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:20.143378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa922ee-eda7-4d3d-b0c8-6d5d7b22037d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:20.143963 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa922ee-eda7-4d3d-b0c8-6d5d7b22037d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:20.143963 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa922ee-eda7-4d3d-b0c8-6d5d7b22037d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:20.145313 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Sep 26 18:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:01:20.151967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d9ddd-622a-4f6c-a916-a4a67a81c4ec None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:20.152487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d9ddd-622a-4f6c-a916-a4a67a81c4ec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:20.152585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d9ddd-622a-4f6c-a916-a4a67a81c4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:20.172478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-343d9ddd-622a-4f6c-a916-a4a67a81c4ec None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:20.173676 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-343d9ddd-622a-4f6c-a916-a4a67a81c4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:20.181556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Sep 26 18:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:20.341686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa922ee-eda7-4d3d-b0c8-6d5d7b22037d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:20.358588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa922ee-eda7-4d3d-b0c8-6d5d7b22037d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:20.365643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c268f-5b54-4e0f-9b71-1c100fec8015 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:20.366363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c268f-5b54-4e0f-9b71-1c100fec8015 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:20.367001 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c268f-5b54-4e0f-9b71-1c100fec8015 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:20.367279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c268f-5b54-4e0f-9b71-1c100fec8015 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:20.377408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c268f-5b54-4e0f-9b71-1c100fec8015 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:20.379551 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c268f-5b54-4e0f-9b71-1c100fec8015 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:20.390216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 763/1524] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:20.723938 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1baa55-6f5b-4196-93bb-3340b5aa8284 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:20.724223 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1baa55-6f5b-4196-93bb-3340b5aa8284 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:20.724496 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1baa55-6f5b-4196-93bb-3340b5aa8284 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:20.725864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 762/1525] 10.222.0.67 () {54 vars in 961 bytes} [Mon Sep 26 18:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:01:20.731035 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29c5f0-a557-4d91-86e5-031ae4c34574 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:20.731329 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29c5f0-a557-4d91-86e5-031ae4c34574 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:20.732239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29c5f0-a557-4d91-86e5-031ae4c34574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:20.751903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8e29c5f0-a557-4d91-86e5-031ae4c34574 None None] MFA Rules not processed for user `22a512e0a64e420abf15b2150ccc2b32`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:20.753104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8e29c5f0-a557-4d91-86e5-031ae4c34574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:20.761959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 764/1526] 10.222.0.67 () {58 vars in 1058 bytes} [Mon Sep 26 18:01:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:20.769831 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1baa55-6f5b-4196-93bb-3340b5aa8284 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:20.786474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1baa55-6f5b-4196-93bb-3340b5aa8284 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:20.791731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0816eb2-0be0-4cb8-9c11-f7c6c020d4cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:20.792345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0816eb2-0be0-4cb8-9c11-f7c6c020d4cf service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:20.792624 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0816eb2-0be0-4cb8-9c11-f7c6c020d4cf service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:20.792896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0816eb2-0be0-4cb8-9c11-f7c6c020d4cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:20.803107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0816eb2-0be0-4cb8-9c11-f7c6c020d4cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:20.804853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0816eb2-0be0-4cb8-9c11-f7c6c020d4cf service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:20.815188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 763/1527] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:24.676096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d72275-edc3-4dd4-8b6a-7bec49f02fdc None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:24.676683 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d72275-edc3-4dd4-8b6a-7bec49f02fdc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:24.676683 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d72275-edc3-4dd4-8b6a-7bec49f02fdc None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:24.678022 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 18:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:01:24.683430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06151a-50cc-4af4-b05e-7cf849a4c026 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:24.683897 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06151a-50cc-4af4-b05e-7cf849a4c026 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:24.684006 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06151a-50cc-4af4-b05e-7cf849a4c026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:24.705908 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9d06151a-50cc-4af4-b05e-7cf849a4c026 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:24.707146 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9d06151a-50cc-4af4-b05e-7cf849a4c026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:24.715318 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 18:01:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:01:25.009517 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d72275-edc3-4dd4-8b6a-7bec49f02fdc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:25.011596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-85d72275-edc3-4dd4-8b6a-7bec49f02fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:25.092437 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d72275-edc3-4dd4-8b6a-7bec49f02fdc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:25.096049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb763e5-ec52-4e9e-9f9d-147e8760b8b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:25.096665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb763e5-ec52-4e9e-9f9d-147e8760b8b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:25.096947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb763e5-ec52-4e9e-9f9d-147e8760b8b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:25.097221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb763e5-ec52-4e9e-9f9d-147e8760b8b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:25.107572 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb763e5-ec52-4e9e-9f9d-147e8760b8b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:25.109055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb763e5-ec52-4e9e-9f9d-147e8760b8b4 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:25.118990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 766/1530] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:27.410721 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b95ba-0f84-4ff2-9633-7ef6f7ca5d16 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:27.411301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b95ba-0f84-4ff2-9633-7ef6f7ca5d16 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:27.411301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b95ba-0f84-4ff2-9633-7ef6f7ca5d16 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:27.412639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 18:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:01:27.417986 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f0859-b8b5-41f5-8fbc-00ba1b2f7785 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:27.418404 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f0859-b8b5-41f5-8fbc-00ba1b2f7785 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:27.418539 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685f0859-b8b5-41f5-8fbc-00ba1b2f7785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:27.438765 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-685f0859-b8b5-41f5-8fbc-00ba1b2f7785 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:27.439986 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-685f0859-b8b5-41f5-8fbc-00ba1b2f7785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:27.447749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 18:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:27.743032 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b95ba-0f84-4ff2-9633-7ef6f7ca5d16 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:27.755275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b95ba-0f84-4ff2-9633-7ef6f7ca5d16 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:27.759233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9a367-3e9b-4404-a3fa-351bb55f57bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:27.759886 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9a367-3e9b-4404-a3fa-351bb55f57bd service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:27.760165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9a367-3e9b-4404-a3fa-351bb55f57bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:27.760442 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9a367-3e9b-4404-a3fa-351bb55f57bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:27.770464 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9a367-3e9b-4404-a3fa-351bb55f57bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:27.771947 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9a367-3e9b-4404-a3fa-351bb55f57bd service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:27.781493 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 766/1533] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:31.707744 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bf0aa-87c7-4e1c-b8bb-481527afc1bd None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:31.708317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bf0aa-87c7-4e1c-b8bb-481527afc1bd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:31.708317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bf0aa-87c7-4e1c-b8bb-481527afc1bd None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:31.710215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 18:01:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:01:31.716016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd556544-96e7-4fa6-8a44-0f2d4ff40bfa None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:31.718688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd556544-96e7-4fa6-8a44-0f2d4ff40bfa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:31.718688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd556544-96e7-4fa6-8a44-0f2d4ff40bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:31.737506 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cd556544-96e7-4fa6-8a44-0f2d4ff40bfa None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:31.738687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cd556544-96e7-4fa6-8a44-0f2d4ff40bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:31.747419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 18:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:32.069131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bf0aa-87c7-4e1c-b8bb-481527afc1bd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:32.070846 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-270bf0aa-87c7-4e1c-b8bb-481527afc1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:32.151109 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bf0aa-87c7-4e1c-b8bb-481527afc1bd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:32.155770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b060ac-08d8-4e49-a7e2-6df209a91f9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:32.156376 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b060ac-08d8-4e49-a7e2-6df209a91f9c service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:32.156656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b060ac-08d8-4e49-a7e2-6df209a91f9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:32.156928 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b060ac-08d8-4e49-a7e2-6df209a91f9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:32.174683 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b060ac-08d8-4e49-a7e2-6df209a91f9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:32.176151 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b060ac-08d8-4e49-a7e2-6df209a91f9c service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:32.186229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 769/1536] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:34.351791 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd5cf7-a9ef-4fb0-8dfe-f5b3e22b2439 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:34.352455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd5cf7-a9ef-4fb0-8dfe-f5b3e22b2439 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:34.352575 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabd5cf7-a9ef-4fb0-8dfe-f5b3e22b2439 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:34.353826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 917 bytes} [Mon Sep 26 18:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:01:34.360114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26f824-e51c-4389-a166-6612cde648b0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:34.360558 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26f824-e51c-4389-a166-6612cde648b0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:34.360653 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f26f824-e51c-4389-a166-6612cde648b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:34.380287 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9f26f824-e51c-4389-a166-6612cde648b0 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:01:34.381531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9f26f824-e51c-4389-a166-6612cde648b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:34.390187 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Mon Sep 26 18:01:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:01:34.694075 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd5cf7-a9ef-4fb0-8dfe-f5b3e22b2439 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:01:34.695784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-aabd5cf7-a9ef-4fb0-8dfe-f5b3e22b2439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:01:34.709342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabd5cf7-a9ef-4fb0-8dfe-f5b3e22b2439 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:01:34.712813 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7896661-cbb1-44f4-a558-91007f8bcff1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:01:34.713400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7896661-cbb1-44f4-a558-91007f8bcff1 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:01:34.713681 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7896661-cbb1-44f4-a558-91007f8bcff1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:01:34.713952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7896661-cbb1-44f4-a558-91007f8bcff1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:01:34.724356 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7896661-cbb1-44f4-a558-91007f8bcff1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:01:34.725781 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7896661-cbb1-44f4-a558-91007f8bcff1 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:01:34.735090 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 769/1539] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:01:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:01:44.375211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:01:44.731138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=33 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:05.815393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2416c9-3f16-4cab-9d1f-d51b0f8fba5f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:05.816066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2416c9-3f16-4cab-9d1f-d51b0f8fba5f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:05.816066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2416c9-3f16-4cab-9d1f-d51b0f8fba5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:05.842263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0f2416c9-3f16-4cab-9d1f-d51b0f8fba5f None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:05.843490 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0f2416c9-3f16-4cab-9d1f-d51b0f8fba5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:05.858095 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 771/1540] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:05.866995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d1418-1da2-4140-bf89-a34a4f570fdc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:05.868453 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d1418-1da2-4140-bf89-a34a4f570fdc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:05.868453 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83d1418-1da2-4140-bf89-a34a4f570fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:05.892485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ede31a-33c7-4339-946d-ade31f7d3d1c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:05.892817 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ede31a-33c7-4339-946d-ade31f7d3d1c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:05.893148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ede31a-33c7-4339-946d-ade31f7d3d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:05.906917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d83d1418-1da2-4140-bf89-a34a4f570fdc None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:05.909960 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d83d1418-1da2-4140-bf89-a34a4f570fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:05.926582 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-91ede31a-33c7-4339-946d-ade31f7d3d1c None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:05.928015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-91ede31a-33c7-4339-946d-ade31f7d3d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:05.933390 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 770/1541] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:05.935181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14909c-3344-4aa4-84ad-e071689c2ee2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:05.935498 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14909c-3344-4aa4-84ad-e071689c2ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:05.935935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a14909c-3344-4aa4-84ad-e071689c2ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:05.943479 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 772/1542] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:05.944488 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ede31a-33c7-4339-946d-ade31f7d3d1c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:05.965927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ede31a-33c7-4339-946d-ade31f7d3d1c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:05.977328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0a14909c-3344-4aa4-84ad-e071689c2ee2 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:05.979380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0a14909c-3344-4aa4-84ad-e071689c2ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:05.986004 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3dc7dd-225a-44be-9793-e772e3bce27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:05.986664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3dc7dd-225a-44be-9793-e772e3bce27d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:05.987023 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3dc7dd-225a-44be-9793-e772e3bce27d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:05.987324 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3dc7dd-225a-44be-9793-e772e3bce27d admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:05.988434 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3dc7dd-225a-44be-9793-e772e3bce27d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:05.988434 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3dc7dd-225a-44be-9793-e772e3bce27d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:05.990181 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3dc7dd-225a-44be-9793-e772e3bce27d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:06.020644 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 771/1543] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:06.022043 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f318d-00e9-468a-805b-fb71a7851922 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.022339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f318d-00e9-468a-805b-fb71a7851922 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.022613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64f318d-00e9-468a-805b-fb71a7851922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.033366 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 773/1544] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:03:06.035149 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ffdeb-1d5f-4f18-a3fa-8df015619388 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.035693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ffdeb-1d5f-4f18-a3fa-8df015619388 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.036198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192ffdeb-1d5f-4f18-a3fa-8df015619388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.054168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b64f318d-00e9-468a-805b-fb71a7851922 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:06.055990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b64f318d-00e9-468a-805b-fb71a7851922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:06.065026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 772/1545] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:06.066109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d384e9-903d-435b-9ed1-2fe16e5f8556 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.066376 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d384e9-903d-435b-9ed1-2fe16e5f8556 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.066591 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d384e9-903d-435b-9ed1-2fe16e5f8556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.067880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-192ffdeb-1d5f-4f18-a3fa-8df015619388 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:06.069840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-192ffdeb-1d5f-4f18-a3fa-8df015619388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:06.080827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 774/1546] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:06.083376 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade0db4-357e-4004-9965-8b8b94325f5c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.083922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade0db4-357e-4004-9965-8b8b94325f5c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.084452 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fade0db4-357e-4004-9965-8b8b94325f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.094761 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-77d384e9-903d-435b-9ed1-2fe16e5f8556 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:06.096009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-77d384e9-903d-435b-9ed1-2fe16e5f8556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:06.105730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 773/1547] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:06.106770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de41a1-17f0-4374-ab8f-f9b863970cc0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.107053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de41a1-17f0-4374-ab8f-f9b863970cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.107323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65de41a1-17f0-4374-ab8f-f9b863970cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.110339 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fade0db4-357e-4004-9965-8b8b94325f5c None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:06.112185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fade0db4-357e-4004-9965-8b8b94325f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:06.128007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 775/1548] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:06.128897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade0db4-357e-4004-9965-8b8b94325f5c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:06.141142 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fade0db4-357e-4004-9965-8b8b94325f5c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:06.146932 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-65de41a1-17f0-4374-ab8f-f9b863970cc0 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:06.147177 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe7a4f-b8f5-45d4-b0c9-619451f67176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:06.147599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe7a4f-b8f5-45d4-b0c9-619451f67176 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.147912 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe7a4f-b8f5-45d4-b0c9-619451f67176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.148227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe7a4f-b8f5-45d4-b0c9-619451f67176 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.148731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-65de41a1-17f0-4374-ab8f-f9b863970cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:06.148926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe7a4f-b8f5-45d4-b0c9-619451f67176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:06.150557 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-33fe7a4f-b8f5-45d4-b0c9-619451f67176 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 18:03:06.150557 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 26 18:03:06.150557 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: Sep 26 18:03:06.151812 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe7a4f-b8f5-45d4-b0c9-619451f67176 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:06.159247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 774/1549] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:06.161782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de41a1-17f0-4374-ab8f-f9b863970cc0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:06.173372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65de41a1-17f0-4374-ab8f-f9b863970cc0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:06.186570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80163a-d2c4-48eb-a541-53ce9759bbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:06.187351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80163a-d2c4-48eb-a541-53ce9759bbe9 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.187633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80163a-d2c4-48eb-a541-53ce9759bbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.187933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80163a-d2c4-48eb-a541-53ce9759bbe9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.188578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80163a-d2c4-48eb-a541-53ce9759bbe9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:06.188875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80163a-d2c4-48eb-a541-53ce9759bbe9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:06.189890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-0c80163a-d2c4-48eb-a541-53ce9759bbe9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 26 18:03:06.189890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Sep 26 18:03:06.189890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: Sep 26 18:03:06.190763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80163a-d2c4-48eb-a541-53ce9759bbe9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:06.204690 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 775/1550] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:03:06.218787 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d364bab-0a9d-4027-8980-0c1530be1882 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.218787 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d364bab-0a9d-4027-8980-0c1530be1882 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.218787 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d364bab-0a9d-4027-8980-0c1530be1882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.226710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0d364bab-0a9d-4027-8980-0c1530be1882 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:06.228197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0d364bab-0a9d-4027-8980-0c1530be1882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:06.236396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 776/1551] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:06.237748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906dfaa-5651-495a-a697-b2790da14241 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.238135 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906dfaa-5651-495a-a697-b2790da14241 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.238510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3906dfaa-5651-495a-a697-b2790da14241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.258701 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3906dfaa-5651-495a-a697-b2790da14241 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:06.264214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3906dfaa-5651-495a-a697-b2790da14241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:06.268416 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 777/1552] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:06.269683 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562558d4-019c-4c7c-ad99-7e00f1707e03 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.269965 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562558d4-019c-4c7c-ad99-7e00f1707e03 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.270236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562558d4-019c-4c7c-ad99-7e00f1707e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.291115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-562558d4-019c-4c7c-ad99-7e00f1707e03 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:06.292370 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-562558d4-019c-4c7c-ad99-7e00f1707e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:06.300308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 778/1553] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:06.301102 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562558d4-019c-4c7c-ad99-7e00f1707e03 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:06.310782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562558d4-019c-4c7c-ad99-7e00f1707e03 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:06.315599 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3050b0d-b0ed-41da-b9f6-0055e6f6fddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:06.316692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3050b0d-b0ed-41da-b9f6-0055e6f6fddc admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.317113 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3050b0d-b0ed-41da-b9f6-0055e6f6fddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.317427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3050b0d-b0ed-41da-b9f6-0055e6f6fddc admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.318080 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3050b0d-b0ed-41da-b9f6-0055e6f6fddc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:06.318387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3050b0d-b0ed-41da-b9f6-0055e6f6fddc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:06.320162 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3050b0d-b0ed-41da-b9f6-0055e6f6fddc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:06.333411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 779/1554] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:03:06.334886 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d6aef-546c-412c-8a30-eed974a0da61 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.335329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d6aef-546c-412c-8a30-eed974a0da61 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.338969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098d6aef-546c-412c-8a30-eed974a0da61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.369275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-098d6aef-546c-412c-8a30-eed974a0da61 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:06.369881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 776/1555] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/projects => generated 403 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:06.370528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-098d6aef-546c-412c-8a30-eed974a0da61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:06.371394 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe7a4f-b8f5-45d4-b0c9-619451f67176 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:06.441339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 780/1556] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:06.442329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d6aef-546c-412c-8a30-eed974a0da61 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:06.450085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe7a4f-b8f5-45d4-b0c9-619451f67176 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:06.453807 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098d6aef-546c-412c-8a30-eed974a0da61 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:06.456312 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51047a56-c81e-448f-bb4d-badb0952d7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:06.457134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51047a56-c81e-448f-bb4d-badb0952d7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.457427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51047a56-c81e-448f-bb4d-badb0952d7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.457829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51047a56-c81e-448f-bb4d-badb0952d7c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.458691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51047a56-c81e-448f-bb4d-badb0952d7c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:06.458822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4479e22-60d0-48a5-99e8-f914562f622a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:06.459432 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4479e22-60d0-48a5-99e8-f914562f622a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.459746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4479e22-60d0-48a5-99e8-f914562f622a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.460261 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4479e22-60d0-48a5-99e8-f914562f622a admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.460903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4479e22-60d0-48a5-99e8-f914562f622a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:06.461232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4479e22-60d0-48a5-99e8-f914562f622a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:06.461385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51047a56-c81e-448f-bb4d-badb0952d7c2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:06.462843 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4479e22-60d0-48a5-99e8-f914562f622a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:06.475588 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 781/1557] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:03:06.476341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4479e22-60d0-48a5-99e8-f914562f622a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:06.486512 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4479e22-60d0-48a5-99e8-f914562f622a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:06.491795 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b34175-205d-41ee-8710-97d6e62e8e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:06.493071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b34175-205d-41ee-8710-97d6e62e8e35 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.494048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b34175-205d-41ee-8710-97d6e62e8e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.494348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b34175-205d-41ee-8710-97d6e62e8e35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.495004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b34175-205d-41ee-8710-97d6e62e8e35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:06.495318 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b34175-205d-41ee-8710-97d6e62e8e35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:06.497028 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b34175-205d-41ee-8710-97d6e62e8e35 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:06.511740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 782/1558] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:03:06.512681 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b34175-205d-41ee-8710-97d6e62e8e35 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:06.522722 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b34175-205d-41ee-8710-97d6e62e8e35 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:06.527963 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603d077-f92f-4b27-86c3-8ad12533611b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:06.528570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603d077-f92f-4b27-86c3-8ad12533611b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.528848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603d077-f92f-4b27-86c3-8ad12533611b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.529124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603d077-f92f-4b27-86c3-8ad12533611b admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.529851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603d077-f92f-4b27-86c3-8ad12533611b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:06.530171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603d077-f92f-4b27-86c3-8ad12533611b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:06.531811 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603d077-f92f-4b27-86c3-8ad12533611b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:06.544435 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 783/1559] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:03:06.545824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ca4fa-ad2f-4505-82d9-2e7665e2e647 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.546114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ca4fa-ad2f-4505-82d9-2e7665e2e647 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.546392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ca4fa-ad2f-4505-82d9-2e7665e2e647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.568091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-901ca4fa-ad2f-4505-82d9-2e7665e2e647 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:06.569193 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-901ca4fa-ad2f-4505-82d9-2e7665e2e647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:06.578215 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 784/1560] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:06.578991 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ca4fa-ad2f-4505-82d9-2e7665e2e647 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:06.588924 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ca4fa-ad2f-4505-82d9-2e7665e2e647 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:06.593539 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cdf23-4832-4061-aa03-fe75be951816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:06.594139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cdf23-4832-4061-aa03-fe75be951816 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.594420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cdf23-4832-4061-aa03-fe75be951816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.594692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cdf23-4832-4061-aa03-fe75be951816 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.595321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cdf23-4832-4061-aa03-fe75be951816 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:06.597252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cdf23-4832-4061-aa03-fe75be951816 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:06.618228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 777/1561] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/projects => generated 375 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:06.618228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51047a56-c81e-448f-bb4d-badb0952d7c2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:06.677781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51047a56-c81e-448f-bb4d-badb0952d7c2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:06.683842 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d848e4-3557-450d-bc75-f6c5331bdfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:06.684487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d848e4-3557-450d-bc75-f6c5331bdfe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.684771 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d848e4-3557-450d-bc75-f6c5331bdfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.685047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d848e4-3557-450d-bc75-f6c5331bdfe9 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.685857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d848e4-3557-450d-bc75-f6c5331bdfe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:06.701974 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d848e4-3557-450d-bc75-f6c5331bdfe9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:06.809436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 785/1562] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/projects => generated 403 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:06.810887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fbbf7-f688-492d-a816-8f269d51d6ca None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.811146 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fbbf7-f688-492d-a816-8f269d51d6ca None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.811264 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7fbbf7-f688-492d-a816-8f269d51d6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.830905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4e7fbbf7-f688-492d-a816-8f269d51d6ca None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:06.832050 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4e7fbbf7-f688-492d-a816-8f269d51d6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:06.841422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:06.847524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 778/1563] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/users => generated 302 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:06.847524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d848e4-3557-450d-bc75-f6c5331bdfe9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:06.850609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:06.855326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d848e4-3557-450d-bc75-f6c5331bdfe9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:06.904579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 786/1564] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:06.905434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7fbbf7-f688-492d-a816-8f269d51d6ca None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:06.915434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7fbbf7-f688-492d-a816-8f269d51d6ca None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:06.920462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6067b-f53c-46fa-b396-2693d0a3191b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:06.921189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6067b-f53c-46fa-b396-2693d0a3191b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.921495 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6067b-f53c-46fa-b396-2693d0a3191b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.921774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6067b-f53c-46fa-b396-2693d0a3191b admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.922536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6067b-f53c-46fa-b396-2693d0a3191b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:06.923487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18811d05-8a8d-40cc-a26f-63069a235860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:06.924139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18811d05-8a8d-40cc-a26f-63069a235860 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:06.924472 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18811d05-8a8d-40cc-a26f-63069a235860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:06.924598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6067b-f53c-46fa-b396-2693d0a3191b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:06.924693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18811d05-8a8d-40cc-a26f-63069a235860 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:06.925340 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18811d05-8a8d-40cc-a26f-63069a235860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:06.927278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18811d05-8a8d-40cc-a26f-63069a235860 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:07.089007 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 787/1565] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/projects => generated 379 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:07.092463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 779/1566] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:06 2022] POST /identity/v3/projects => generated 373 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:07.093107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6067b-f53c-46fa-b396-2693d0a3191b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:07.093866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bd8e0-aff8-4b3c-8e38-819cba31c21f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:07.094155 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bd8e0-aff8-4b3c-8e38-819cba31c21f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:07.094432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8bd8e0-aff8-4b3c-8e38-819cba31c21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:07.115075 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2d8bd8e0-aff8-4b3c-8e38-819cba31c21f None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:07.116466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2d8bd8e0-aff8-4b3c-8e38-819cba31c21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:07.159484 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6067b-f53c-46fa-b396-2693d0a3191b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:07.177545 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 780/1567] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:07.178480 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bd8e0-aff8-4b3c-8e38-819cba31c21f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:07.188728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8bd8e0-aff8-4b3c-8e38-819cba31c21f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:07.193387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c8348-0e96-40af-a4c3-7c8c76e2b0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:07.193990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c8348-0e96-40af-a4c3-7c8c76e2b0c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:07.194277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c8348-0e96-40af-a4c3-7c8c76e2b0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:07.194552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c8348-0e96-40af-a4c3-7c8c76e2b0c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:07.195184 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c8348-0e96-40af-a4c3-7c8c76e2b0c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:07.197473 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c8348-0e96-40af-a4c3-7c8c76e2b0c7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:07.218435 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60a95d-1c68-4930-9a5a-5f218d43b00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:07.219139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60a95d-1c68-4930-9a5a-5f218d43b00f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:07.219427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60a95d-1c68-4930-9a5a-5f218d43b00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:07.219805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60a95d-1c68-4930-9a5a-5f218d43b00f admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:07.220441 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60a95d-1c68-4930-9a5a-5f218d43b00f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:07.222414 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60a95d-1c68-4930-9a5a-5f218d43b00f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:07.353990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 781/1568] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:07 2022] POST /identity/v3/users => generated 288 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:07.353990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c8348-0e96-40af-a4c3-7c8c76e2b0c7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:07.361832 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c8348-0e96-40af-a4c3-7c8c76e2b0c7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:07.366811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e18d4f-ac66-4269-a120-f23cc6bf4834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:07.367465 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e18d4f-ac66-4269-a120-f23cc6bf4834 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:07.367846 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e18d4f-ac66-4269-a120-f23cc6bf4834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:07.368098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e18d4f-ac66-4269-a120-f23cc6bf4834 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:07.368728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e18d4f-ac66-4269-a120-f23cc6bf4834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:07.370655 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e18d4f-ac66-4269-a120-f23cc6bf4834 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:07.448476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 788/1569] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:07 2022] POST /identity/v3/projects => generated 383 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:07.471103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60a95d-1c68-4930-9a5a-5f218d43b00f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:07.478587 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:07.511135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 782/1570] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:07 2022] POST /identity/v3/users => generated 302 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:07.511135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e18d4f-ac66-4269-a120-f23cc6bf4834 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:07.515366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60a95d-1c68-4930-9a5a-5f218d43b00f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:07.519785 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e18d4f-ac66-4269-a120-f23cc6bf4834 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:07.520574 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad55977-2650-4036-bace-c06820a1e11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:07.521185 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad55977-2650-4036-bace-c06820a1e11c admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:07.521473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad55977-2650-4036-bace-c06820a1e11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:07.521873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad55977-2650-4036-bace-c06820a1e11c admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:07.522521 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad55977-2650-4036-bace-c06820a1e11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:07.524784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad55977-2650-4036-bace-c06820a1e11c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:07.526696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77db49-baa1-45c7-8b4c-74867d21d163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:07.527386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77db49-baa1-45c7-8b4c-74867d21d163 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:07.527784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77db49-baa1-45c7-8b4c-74867d21d163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:07.528061 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab77db49-baa1-45c7-8b4c-74867d21d163 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:07.563071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77db49-baa1-45c7-8b4c-74867d21d163 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:07.563071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77db49-baa1-45c7-8b4c-74867d21d163 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:07.563071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab77db49-baa1-45c7-8b4c-74867d21d163 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:07.563420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 789/1571] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:07.563420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad55977-2650-4036-bace-c06820a1e11c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:07.563611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 783/1572] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:03:07.563611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77db49-baa1-45c7-8b4c-74867d21d163 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:07.563823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad55977-2650-4036-bace-c06820a1e11c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:07.563823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae4e1e-ddf9-4b1a-9a55-131ce5f1f505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:07.563823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae4e1e-ddf9-4b1a-9a55-131ce5f1f505 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:07.563823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae4e1e-ddf9-4b1a-9a55-131ce5f1f505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:07.564161 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cae4e1e-ddf9-4b1a-9a55-131ce5f1f505 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:07.564254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab77db49-baa1-45c7-8b4c-74867d21d163 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:07.564346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae4e1e-ddf9-4b1a-9a55-131ce5f1f505 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:07.564346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-3cae4e1e-ddf9-4b1a-9a55-131ce5f1f505 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 18:03:07.564346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Sep 26 18:03:07.564346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: Sep 26 18:03:07.564346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cae4e1e-ddf9-4b1a-9a55-131ce5f1f505 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:07.568616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d976ccf-de10-43f0-9cc0-d922a3ff3f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:07.569239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d976ccf-de10-43f0-9cc0-d922a3ff3f42 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:07.569569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d976ccf-de10-43f0-9cc0-d922a3ff3f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:07.569862 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d976ccf-de10-43f0-9cc0-d922a3ff3f42 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:07.571020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d976ccf-de10-43f0-9cc0-d922a3ff3f42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:07.573914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d976ccf-de10-43f0-9cc0-d922a3ff3f42 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:07.805543 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 784/1573] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:07 2022] POST /identity/v3/users => generated 287 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:07.806218 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 790/1574] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:07 2022] POST /identity/v3/users => generated 290 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:07.807153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d976ccf-de10-43f0-9cc0-d922a3ff3f42 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:07.807798 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae4e1e-ddf9-4b1a-9a55-131ce5f1f505 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:07.818664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d976ccf-de10-43f0-9cc0-d922a3ff3f42 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:07.823532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cae4e1e-ddf9-4b1a-9a55-131ce5f1f505 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:07.824220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe487840-46bd-409a-9646-7dffec6e99c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:07.824828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe487840-46bd-409a-9646-7dffec6e99c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:07.825116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe487840-46bd-409a-9646-7dffec6e99c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:07.825381 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe487840-46bd-409a-9646-7dffec6e99c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:07.826026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe487840-46bd-409a-9646-7dffec6e99c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:07.826330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe487840-46bd-409a-9646-7dffec6e99c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:07.827970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe487840-46bd-409a-9646-7dffec6e99c0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:07.828739 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfe514-7316-41b9-9dbd-00b899ce11f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:07.829356 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfe514-7316-41b9-9dbd-00b899ce11f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:07.829857 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfe514-7316-41b9-9dbd-00b899ce11f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:07.830138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dfe514-7316-41b9-9dbd-00b899ce11f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:07.831346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfe514-7316-41b9-9dbd-00b899ce11f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:07.832991 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dfe514-7316-41b9-9dbd-00b899ce11f6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:07.841509 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 785/1575] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:03:07.842101 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe487840-46bd-409a-9646-7dffec6e99c0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:07.847213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 791/1576] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:07.849042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfe514-7316-41b9-9dbd-00b899ce11f6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:07.854548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe487840-46bd-409a-9646-7dffec6e99c0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:07.859284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc7227-14d0-4110-9313-e1768d31bed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:07.859964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc7227-14d0-4110-9313-e1768d31bed9 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:07.860244 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc7227-14d0-4110-9313-e1768d31bed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:07.860515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc7227-14d0-4110-9313-e1768d31bed9 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:07.861150 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc7227-14d0-4110-9313-e1768d31bed9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:07.862783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dfe514-7316-41b9-9dbd-00b899ce11f6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:07.863350 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc7227-14d0-4110-9313-e1768d31bed9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:07.867978 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546d87d-1e52-4d4c-bea0-c7a3ac4f270e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:07.868624 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546d87d-1e52-4d4c-bea0-c7a3ac4f270e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:07.868908 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546d87d-1e52-4d4c-bea0-c7a3ac4f270e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:07.869186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6546d87d-1e52-4d4c-bea0-c7a3ac4f270e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:07.869834 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546d87d-1e52-4d4c-bea0-c7a3ac4f270e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:07.871473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6546d87d-1e52-4d4c-bea0-c7a3ac4f270e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:07.874260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:07.884638 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 792/1577] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:07.886202 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546d87d-1e52-4d4c-bea0-c7a3ac4f270e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:07.899342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6546d87d-1e52-4d4c-bea0-c7a3ac4f270e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:07.904799 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf64611-ee06-4260-85a9-2ae690b0d71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:07.905441 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf64611-ee06-4260-85a9-2ae690b0d71e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:07.905723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf64611-ee06-4260-85a9-2ae690b0d71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:07.905998 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf64611-ee06-4260-85a9-2ae690b0d71e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:07.906654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf64611-ee06-4260-85a9-2ae690b0d71e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:07.908336 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf64611-ee06-4260-85a9-2ae690b0d71e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:07.919085 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 793/1578] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:07.920070 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf64611-ee06-4260-85a9-2ae690b0d71e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:07.930500 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf64611-ee06-4260-85a9-2ae690b0d71e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:07.935810 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407f2c3-8e76-4f39-831b-043519f8f538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:07.936667 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407f2c3-8e76-4f39-831b-043519f8f538 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:07.936989 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407f2c3-8e76-4f39-831b-043519f8f538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:07.944438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407f2c3-8e76-4f39-831b-043519f8f538 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:07.944438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407f2c3-8e76-4f39-831b-043519f8f538 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:07.944438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407f2c3-8e76-4f39-831b-043519f8f538 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:08.107603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 786/1579] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:07 2022] POST /identity/v3/users => generated 292 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:08.108720 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc7227-14d0-4110-9313-e1768d31bed9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:08.120055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc7227-14d0-4110-9313-e1768d31bed9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:08.125642 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c6333-7d14-4bfb-968e-3ae6e1ed33b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:08.126260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c6333-7d14-4bfb-968e-3ae6e1ed33b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:08.126536 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c6333-7d14-4bfb-968e-3ae6e1ed33b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:08.126805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732c6333-7d14-4bfb-968e-3ae6e1ed33b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:08.129270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c6333-7d14-4bfb-968e-3ae6e1ed33b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:08.131945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732c6333-7d14-4bfb-968e-3ae6e1ed33b9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:08.146852 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 794/1580] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:07 2022] POST /identity/v3/projects => generated 393 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:08.151032 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 787/1581] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:08.151149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407f2c3-8e76-4f39-831b-043519f8f538 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:08.151257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c6333-7d14-4bfb-968e-3ae6e1ed33b9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:08.160037 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732c6333-7d14-4bfb-968e-3ae6e1ed33b9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:08.225380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5f54e-7813-4a07-ace8-80b09a882d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:08.226031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5f54e-7813-4a07-ace8-80b09a882d55 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:08.226315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5f54e-7813-4a07-ace8-80b09a882d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:08.226592 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5f54e-7813-4a07-ace8-80b09a882d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:08.227317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5f54e-7813-4a07-ace8-80b09a882d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:08.228653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407f2c3-8e76-4f39-831b-043519f8f538 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:08.228998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5f54e-7813-4a07-ace8-80b09a882d55 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:08.234591 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc0f61-ede7-405d-9a55-1e25ab7b7507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:08.235215 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc0f61-ede7-405d-9a55-1e25ab7b7507 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:08.235519 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc0f61-ede7-405d-9a55-1e25ab7b7507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:08.235841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc0f61-ede7-405d-9a55-1e25ab7b7507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:08.236473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc0f61-ede7-405d-9a55-1e25ab7b7507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:08.238088 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc0f61-ede7-405d-9a55-1e25ab7b7507 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:08.240152 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 788/1582] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:08.241383 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5f54e-7813-4a07-ace8-80b09a882d55 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:08.254002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 795/1583] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:08.254002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc0f61-ede7-405d-9a55-1e25ab7b7507 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:08.256761 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5f54e-7813-4a07-ace8-80b09a882d55 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:08.262833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2312b-2231-4b02-b08a-ffd2d6ce809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:08.263404 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc0f61-ede7-405d-9a55-1e25ab7b7507 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:08.263511 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2312b-2231-4b02-b08a-ffd2d6ce809f admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:08.263761 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2312b-2231-4b02-b08a-ffd2d6ce809f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:08.264036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2312b-2231-4b02-b08a-ffd2d6ce809f admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:08.264825 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2312b-2231-4b02-b08a-ffd2d6ce809f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:08.266817 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e2312b-2231-4b02-b08a-ffd2d6ce809f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:08.269215 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479571a-4481-4eee-8b9b-66cfac445c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:08.269832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479571a-4481-4eee-8b9b-66cfac445c7a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:08.270115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479571a-4481-4eee-8b9b-66cfac445c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:08.270389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1479571a-4481-4eee-8b9b-66cfac445c7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:08.271037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479571a-4481-4eee-8b9b-66cfac445c7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:08.272673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1479571a-4481-4eee-8b9b-66cfac445c7a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:08.287698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 796/1584] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:08.287698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479571a-4481-4eee-8b9b-66cfac445c7a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:08.297274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1479571a-4481-4eee-8b9b-66cfac445c7a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:08.302085 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27185049-d13b-4a39-8416-d07d3a0fca5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:08.302772 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27185049-d13b-4a39-8416-d07d3a0fca5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:08.305888 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27185049-d13b-4a39-8416-d07d3a0fca5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:08.305888 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27185049-d13b-4a39-8416-d07d3a0fca5b admin admin] PATH_INFO: `/v3/projects/2054a4e806234580bd7d4e4ac5e97924/users/d652e96acb0545e591baec8551ccae63/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:08.322525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27185049-d13b-4a39-8416-d07d3a0fca5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2054a4e806234580bd7d4e4ac5e97924, user_id=d652e96acb0545e591baec8551ccae63, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:08.325455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27185049-d13b-4a39-8416-d07d3a0fca5b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:08.387700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 789/1585] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:08 2022] POST /identity/v3/projects => generated 379 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:08.389786 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2312b-2231-4b02-b08a-ffd2d6ce809f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:08.425907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 797/1586] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:08 2022] PUT /identity/v3/projects/2054a4e806234580bd7d4e4ac5e97924/users/d652e96acb0545e591baec8551ccae63/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:08.426814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27185049-d13b-4a39-8416-d07d3a0fca5b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:08.450824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:08.465683 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e2312b-2231-4b02-b08a-ffd2d6ce809f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:08.511547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27185049-d13b-4a39-8416-d07d3a0fca5b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:08.518919 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8756a74-feca-4aab-9316-bf2618d8389f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:08.519743 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8756a74-feca-4aab-9316-bf2618d8389f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:08.520026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8756a74-feca-4aab-9316-bf2618d8389f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:08.520415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8756a74-feca-4aab-9316-bf2618d8389f admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:08.521057 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8756a74-feca-4aab-9316-bf2618d8389f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:08.526214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8756a74-feca-4aab-9316-bf2618d8389f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:08.536038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6b3eb-3c0f-4545-b9af-5c4d7e624505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:08.536822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6b3eb-3c0f-4545-b9af-5c4d7e624505 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:08.537112 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6b3eb-3c0f-4545-b9af-5c4d7e624505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:08.537377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6b3eb-3c0f-4545-b9af-5c4d7e624505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:08.538051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6b3eb-3c0f-4545-b9af-5c4d7e624505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:08.539893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b6b3eb-3c0f-4545-b9af-5c4d7e624505 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:08.545921 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:08.551363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 790/1587] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:08.557869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6b3eb-3c0f-4545-b9af-5c4d7e624505 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:08.562853 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6b3eb-3c0f-4545-b9af-5c4d7e624505 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:08.568026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa7bed-ab10-4e31-926c-c59b16310d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:08.568635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa7bed-ab10-4e31-926c-c59b16310d5e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:08.568914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa7bed-ab10-4e31-926c-c59b16310d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:08.569312 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa7bed-ab10-4e31-926c-c59b16310d5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:08.569955 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa7bed-ab10-4e31-926c-c59b16310d5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:08.572432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa7bed-ab10-4e31-926c-c59b16310d5e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:08.584682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 791/1588] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:08.592303 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa7bed-ab10-4e31-926c-c59b16310d5e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:08.597990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa7bed-ab10-4e31-926c-c59b16310d5e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:08.604100 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21e322-c46c-4527-8ae4-b6ce3cff1b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:08.604764 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21e322-c46c-4527-8ae4-b6ce3cff1b13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:08.605045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21e322-c46c-4527-8ae4-b6ce3cff1b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:08.605317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21e322-c46c-4527-8ae4-b6ce3cff1b13 admin admin] PATH_INFO: `/v3/projects/247ce86198ec46aa9a566403ccefaa28/users/6999c1f707f14c89b1556cdf0f886b53/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:08.623276 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21e322-c46c-4527-8ae4-b6ce3cff1b13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=247ce86198ec46aa9a566403ccefaa28, user_id=6999c1f707f14c89b1556cdf0f886b53, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:08.624345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-1a21e322-c46c-4527-8ae4-b6ce3cff1b13 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 18:03:08.624345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 26 18:03:08.624345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: Sep 26 18:03:08.626927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21e322-c46c-4527-8ae4-b6ce3cff1b13 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:08.673900 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 798/1589] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:08 2022] POST /identity/v3/users => generated 297 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:08.674948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8756a74-feca-4aab-9316-bf2618d8389f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:08.685602 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8756a74-feca-4aab-9316-bf2618d8389f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:08.691562 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b763a-7a07-402a-b8c0-e2fa952f6c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:08.692646 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b763a-7a07-402a-b8c0-e2fa952f6c4f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:08.693229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b763a-7a07-402a-b8c0-e2fa952f6c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:08.693684 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b763a-7a07-402a-b8c0-e2fa952f6c4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:08.694764 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b763a-7a07-402a-b8c0-e2fa952f6c4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:08.698254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489b763a-7a07-402a-b8c0-e2fa952f6c4f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:08.710001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 799/1590] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:08.711945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b763a-7a07-402a-b8c0-e2fa952f6c4f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:08.721469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b763a-7a07-402a-b8c0-e2fa952f6c4f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:08.726205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75120f37-51e5-4951-b4af-a1e62abc12b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:08.726949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75120f37-51e5-4951-b4af-a1e62abc12b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:08.727327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75120f37-51e5-4951-b4af-a1e62abc12b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:08.727816 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75120f37-51e5-4951-b4af-a1e62abc12b3 admin admin] PATH_INFO: `/v3/projects/0f9a34cf9c6e44f99b352c5a45dce759/users/ed9c5ada4eda4254a061acdfcc19f4e1/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:08.744511 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75120f37-51e5-4951-b4af-a1e62abc12b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f9a34cf9c6e44f99b352c5a45dce759, user_id=ed9c5ada4eda4254a061acdfcc19f4e1, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:08.752088 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75120f37-51e5-4951-b4af-a1e62abc12b3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:08.780516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 792/1591] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:08 2022] PUT /identity/v3/projects/247ce86198ec46aa9a566403ccefaa28/users/6999c1f707f14c89b1556cdf0f886b53/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:08.781793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21e322-c46c-4527-8ae4-b6ce3cff1b13 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:08.852227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21e322-c46c-4527-8ae4-b6ce3cff1b13 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:08.857737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 800/1592] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:08 2022] PUT /identity/v3/projects/0f9a34cf9c6e44f99b352c5a45dce759/users/ed9c5ada4eda4254a061acdfcc19f4e1/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:08.860268 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c163a-7c12-4385-8648-9bf6d124a274 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:08.860829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c163a-7c12-4385-8648-9bf6d124a274 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:08.861315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c163a-7c12-4385-8648-9bf6d124a274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:08.896904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-de5c163a-7c12-4385-8648-9bf6d124a274 None None] MFA Rules not processed for user `d652e96acb0545e591baec8551ccae63`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:08.898211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-de5c163a-7c12-4385-8648-9bf6d124a274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:08.918580 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c461e7-5d17-458b-a0a1-66b59847c7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:08.919485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c461e7-5d17-458b-a0a1-66b59847c7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:08.919728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c461e7-5d17-458b-a0a1-66b59847c7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:08.919919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c461e7-5d17-458b-a0a1-66b59847c7a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:08.921015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c461e7-5d17-458b-a0a1-66b59847c7a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:08.923277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c461e7-5d17-458b-a0a1-66b59847c7a4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:08.969032 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 801/1593] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:08 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:08.969981 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c163a-7c12-4385-8648-9bf6d124a274 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:08.982098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c163a-7c12-4385-8648-9bf6d124a274 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:08.986890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60534d86-809b-40df-9740-f49bfc851e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:08.987555 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60534d86-809b-40df-9740-f49bfc851e9c admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:08.987870 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60534d86-809b-40df-9740-f49bfc851e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:08.989832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60534d86-809b-40df-9740-f49bfc851e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:08.989832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60534d86-809b-40df-9740-f49bfc851e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:08.990551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60534d86-809b-40df-9740-f49bfc851e9c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:09.001872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 802/1594] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:09.003002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60534d86-809b-40df-9740-f49bfc851e9c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:09.025340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60534d86-809b-40df-9740-f49bfc851e9c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:09.034255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94a1114-e4b7-44e3-9471-8ae5d37669d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:09.034926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a1114-e4b7-44e3-9471-8ae5d37669d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.035201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a1114-e4b7-44e3-9471-8ae5d37669d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.035470 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94a1114-e4b7-44e3-9471-8ae5d37669d6 admin admin] PATH_INFO: `/v3/projects/8e000e309ef2471096d87b38ea3c8cb6/users/ba3c05f195ee47148189a400cc1ac118/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.051958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a1114-e4b7-44e3-9471-8ae5d37669d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e000e309ef2471096d87b38ea3c8cb6, user_id=ba3c05f195ee47148189a400cc1ac118, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:09.054606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94a1114-e4b7-44e3-9471-8ae5d37669d6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:09.064793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 793/1595] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:08 2022] POST /identity/v3/users => generated 290 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:09.065672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c461e7-5d17-458b-a0a1-66b59847c7a4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:09.075830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c461e7-5d17-458b-a0a1-66b59847c7a4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:09.080457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f6911-1154-4859-860f-0c2f1d835a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:09.081194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f6911-1154-4859-860f-0c2f1d835a0a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.081474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f6911-1154-4859-860f-0c2f1d835a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.081748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f6911-1154-4859-860f-0c2f1d835a0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.082412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f6911-1154-4859-860f-0c2f1d835a0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:09.084053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6f6911-1154-4859-860f-0c2f1d835a0a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:09.094596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 794/1596] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:09.095827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f6911-1154-4859-860f-0c2f1d835a0a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:09.106132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f6911-1154-4859-860f-0c2f1d835a0a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:09.110769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3432d559-f738-48fb-90e1-30511d534795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:09.111414 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432d559-f738-48fb-90e1-30511d534795 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.111749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432d559-f738-48fb-90e1-30511d534795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.112043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3432d559-f738-48fb-90e1-30511d534795 admin admin] PATH_INFO: `/v3/projects/991f7c77d7734c56a3a22458ce7dc601/users/b06193cf39004cc9b1fa37f6bf9de115/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.128213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432d559-f738-48fb-90e1-30511d534795 admin admin] RBAC: Authorizing `identity:create_grant(project_id=991f7c77d7734c56a3a22458ce7dc601, user_id=b06193cf39004cc9b1fa37f6bf9de115, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:09.131078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3432d559-f738-48fb-90e1-30511d534795 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:09.188244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 803/1597] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:09 2022] PUT /identity/v3/projects/8e000e309ef2471096d87b38ea3c8cb6/users/ba3c05f195ee47148189a400cc1ac118/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:09.189489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628ed59-b5e6-48c8-b787-f0a8b65da840 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.189770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628ed59-b5e6-48c8-b787-f0a8b65da840 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.190040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d628ed59-b5e6-48c8-b787-f0a8b65da840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.220957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d628ed59-b5e6-48c8-b787-f0a8b65da840 None None] MFA Rules not processed for user `6999c1f707f14c89b1556cdf0f886b53`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:09.222078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d628ed59-b5e6-48c8-b787-f0a8b65da840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:09.227806 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:09.237145 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 795/1598] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:09 2022] PUT /identity/v3/projects/991f7c77d7734c56a3a22458ce7dc601/users/b06193cf39004cc9b1fa37f6bf9de115/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:09.238862 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f50ccc-148d-4645-929e-b3bb1d4e0570 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.239190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f50ccc-148d-4645-929e-b3bb1d4e0570 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.239457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f50ccc-148d-4645-929e-b3bb1d4e0570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.263798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-65f50ccc-148d-4645-929e-b3bb1d4e0570 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:09.265063 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-65f50ccc-148d-4645-929e-b3bb1d4e0570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:09.292753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:09.315191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 804/1599] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:09.316222 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628ed59-b5e6-48c8-b787-f0a8b65da840 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:09.371338 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 796/1600] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:09.373494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f50ccc-148d-4645-929e-b3bb1d4e0570 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:09.392408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f50ccc-148d-4645-929e-b3bb1d4e0570 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:09.397883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0d08e-1881-4506-8b1e-4799ab48ade7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:09.398640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0d08e-1881-4506-8b1e-4799ab48ade7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.398930 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0d08e-1881-4506-8b1e-4799ab48ade7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.399211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0d08e-1881-4506-8b1e-4799ab48ade7 admin admin] PATH_INFO: `/v3/projects/ffa58ff5197e48709d5c90ec334b9403/users/0bede3a601f941e7abc9fd16983679b2/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.415508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0d08e-1881-4506-8b1e-4799ab48ade7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffa58ff5197e48709d5c90ec334b9403, user_id=0bede3a601f941e7abc9fd16983679b2, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:09.416498 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d628ed59-b5e6-48c8-b787-f0a8b65da840 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:09.418224 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0d08e-1881-4506-8b1e-4799ab48ade7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:09.421196 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f712b-9bd3-4a73-9b2e-8abf86d3e24b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:09.421800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f712b-9bd3-4a73-9b2e-8abf86d3e24b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.422118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f712b-9bd3-4a73-9b2e-8abf86d3e24b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.422735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293f712b-9bd3-4a73-9b2e-8abf86d3e24b admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.423458 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f712b-9bd3-4a73-9b2e-8abf86d3e24b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:09.426572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293f712b-9bd3-4a73-9b2e-8abf86d3e24b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:09.594379 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 797/1601] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:09 2022] PUT /identity/v3/projects/ffa58ff5197e48709d5c90ec334b9403/users/0bede3a601f941e7abc9fd16983679b2/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:09.595227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0d08e-1881-4506-8b1e-4799ab48ade7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:09.623779 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 805/1602] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:09 2022] POST /identity/v3/projects => generated 405 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:09.626014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f712b-9bd3-4a73-9b2e-8abf86d3e24b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:09.670560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0d08e-1881-4506-8b1e-4799ab48ade7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:09.696115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293f712b-9bd3-4a73-9b2e-8abf86d3e24b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:09.703551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9a12e3-c145-45fa-9cea-6ee535a4d32f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:09.704189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a12e3-c145-45fa-9cea-6ee535a4d32f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.704470 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a12e3-c145-45fa-9cea-6ee535a4d32f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.704739 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9a12e3-c145-45fa-9cea-6ee535a4d32f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.705394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a12e3-c145-45fa-9cea-6ee535a4d32f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:09.707009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9a12e3-c145-45fa-9cea-6ee535a4d32f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:09.717994 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 806/1603] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:09.719585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a7bdf-310e-4034-928f-4bd8fdaab8f5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.719896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a7bdf-310e-4034-928f-4bd8fdaab8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.720172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49a7bdf-310e-4034-928f-4bd8fdaab8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.743693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f13f6f-820f-4c2c-85f0-50e6174e7fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:09.744319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f13f6f-820f-4c2c-85f0-50e6174e7fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.744603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f13f6f-820f-4c2c-85f0-50e6174e7fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.744880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f13f6f-820f-4c2c-85f0-50e6174e7fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.745534 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f13f6f-820f-4c2c-85f0-50e6174e7fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:09.747143 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f13f6f-820f-4c2c-85f0-50e6174e7fd1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:09.750523 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e49a7bdf-310e-4034-928f-4bd8fdaab8f5 None None] MFA Rules not processed for user `ba3c05f195ee47148189a400cc1ac118`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:09.752036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e49a7bdf-310e-4034-928f-4bd8fdaab8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:09.761831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 798/1604] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:09.761831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e922e-b8e0-40fe-a5d8-2a85102f881e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.761831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e922e-b8e0-40fe-a5d8-2a85102f881e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.761831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170e922e-b8e0-40fe-a5d8-2a85102f881e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.788775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-170e922e-b8e0-40fe-a5d8-2a85102f881e None None] MFA Rules not processed for user `b06193cf39004cc9b1fa37f6bf9de115`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:09.792127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-170e922e-b8e0-40fe-a5d8-2a85102f881e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:09.826610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 807/1605] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:09.828170 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2d7df-4da2-4f09-9f07-f37d57f0a400 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.828455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2d7df-4da2-4f09-9f07-f37d57f0a400 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.828747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2d7df-4da2-4f09-9f07-f37d57f0a400 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.830246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 808/1606] 10.222.0.67 () {54 vars in 961 bytes} [Mon Sep 26 18:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:03:09.830891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2d7df-4da2-4f09-9f07-f37d57f0a400 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:09.866053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 799/1607] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:09.868239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468defd-c221-4c57-8d2b-31f4c6ed271d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.868917 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468defd-c221-4c57-8d2b-31f4c6ed271d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.869351 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e468defd-c221-4c57-8d2b-31f4c6ed271d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.887215 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:09.890136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e468defd-c221-4c57-8d2b-31f4c6ed271d None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:09.891375 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e468defd-c221-4c57-8d2b-31f4c6ed271d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:09.898410 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2d7df-4da2-4f09-9f07-f37d57f0a400 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:09.903617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 800/1608] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:09.903617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468defd-c221-4c57-8d2b-31f4c6ed271d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:09.903891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151e444-fd11-4070-8a6a-fb586c1538ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:09.903891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151e444-fd11-4070-8a6a-fb586c1538ad service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.904068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151e444-fd11-4070-8a6a-fb586c1538ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.904262 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a151e444-fd11-4070-8a6a-fb586c1538ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.910521 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e468defd-c221-4c57-8d2b-31f4c6ed271d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:09.915369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151e444-fd11-4070-8a6a-fb586c1538ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:09.916749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a151e444-fd11-4070-8a6a-fb586c1538ad service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:09.917415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719129ec-a64e-47d7-bbed-3307d02f3e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:09.918100 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719129ec-a64e-47d7-bbed-3307d02f3e70 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.918382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719129ec-a64e-47d7-bbed-3307d02f3e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.918656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719129ec-a64e-47d7-bbed-3307d02f3e70 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.919292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719129ec-a64e-47d7-bbed-3307d02f3e70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:09.921239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719129ec-a64e-47d7-bbed-3307d02f3e70 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:09.927257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 809/1609] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:09.928257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151e444-fd11-4070-8a6a-fb586c1538ad service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:09.938078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a151e444-fd11-4070-8a6a-fb586c1538ad service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:09.942878 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c6a6f-a654-4b35-a61d-a0e0abf08951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:09.943537 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c6a6f-a654-4b35-a61d-a0e0abf08951 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:09.943846 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c6a6f-a654-4b35-a61d-a0e0abf08951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:09.944118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5c6a6f-a654-4b35-a61d-a0e0abf08951 admin admin] PATH_INFO: `/v3/projects/3a15ebd862054a64ac1d739db5adc3a2/users/97ed136f68c540abbdaf04cbe34a682e/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:09.959383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c6a6f-a654-4b35-a61d-a0e0abf08951 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a15ebd862054a64ac1d739db5adc3a2, user_id=97ed136f68c540abbdaf04cbe34a682e, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:09.963832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5c6a6f-a654-4b35-a61d-a0e0abf08951 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:10.211440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 801/1610] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:09 2022] POST /identity/v3/users => generated 309 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:10.212490 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719129ec-a64e-47d7-bbed-3307d02f3e70 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:10.222213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719129ec-a64e-47d7-bbed-3307d02f3e70 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:10.290279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:10.290279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060fd33d-a2d3-43a3-9393-2e7d3c9aa52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:10.290279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fd33d-a2d3-43a3-9393-2e7d3c9aa52c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:10.290279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fd33d-a2d3-43a3-9393-2e7d3c9aa52c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:10.290690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060fd33d-a2d3-43a3-9393-2e7d3c9aa52c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:10.290690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fd33d-a2d3-43a3-9393-2e7d3c9aa52c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:10.290690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060fd33d-a2d3-43a3-9393-2e7d3c9aa52c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:10.290981 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 810/1611] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:09 2022] PUT /identity/v3/projects/3a15ebd862054a64ac1d739db5adc3a2/users/97ed136f68c540abbdaf04cbe34a682e/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:10.290981 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c6a6f-a654-4b35-a61d-a0e0abf08951 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:10.291495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 802/1612] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:10.291495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cec41d-e9ac-4655-a093-799dd66349ab None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:10.291495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cec41d-e9ac-4655-a093-799dd66349ab None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:10.291495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cec41d-e9ac-4655-a093-799dd66349ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:10.292473 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-84cec41d-e9ac-4655-a093-799dd66349ab None None] MFA Rules not processed for user `b9ed29a3f6be4d64bb3751f9e95eb56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:10.293933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-84cec41d-e9ac-4655-a093-799dd66349ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:10.328489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5c6a6f-a654-4b35-a61d-a0e0abf08951 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:10.338358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714189b-77ea-41f6-bbbf-511150984852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:10.338358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714189b-77ea-41f6-bbbf-511150984852 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:10.338358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714189b-77ea-41f6-bbbf-511150984852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:10.338358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5714189b-77ea-41f6-bbbf-511150984852 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:10.339231 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714189b-77ea-41f6-bbbf-511150984852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:10.339231 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5714189b-77ea-41f6-bbbf-511150984852 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:10.393864 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 803/1613] 10.222.0.67 () {58 vars in 1058 bytes} [Mon Sep 26 18:03:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:10.394710 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cec41d-e9ac-4655-a093-799dd66349ab None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:10.405944 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cec41d-e9ac-4655-a093-799dd66349ab None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:10.411468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3eb71-a03f-4978-b143-7b2baafa7c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:10.412120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3eb71-a03f-4978-b143-7b2baafa7c80 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:10.412400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3eb71-a03f-4978-b143-7b2baafa7c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:10.412670 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f3eb71-a03f-4978-b143-7b2baafa7c80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:10.413302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3eb71-a03f-4978-b143-7b2baafa7c80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:10.415965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f3eb71-a03f-4978-b143-7b2baafa7c80 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:10.470951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 811/1614] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:10 2022] POST /identity/v3/projects => generated 371 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:10.471803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714189b-77ea-41f6-bbbf-511150984852 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:10.487559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5714189b-77ea-41f6-bbbf-511150984852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:10.562849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5714189b-77ea-41f6-bbbf-511150984852 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:10.567675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9ae0cb-023d-469e-abd6-dfee207ccd39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:10.568359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ae0cb-023d-469e-abd6-dfee207ccd39 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:10.568714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ae0cb-023d-469e-abd6-dfee207ccd39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:10.568993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9ae0cb-023d-469e-abd6-dfee207ccd39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:10.629617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 804/1615] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:10 2022] POST /identity/v3/projects => generated 381 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:10.630730 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3eb71-a03f-4978-b143-7b2baafa7c80 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:10.643467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ae0cb-023d-469e-abd6-dfee207ccd39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:10.644953 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9ae0cb-023d-469e-abd6-dfee207ccd39 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:10.657267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 812/1616] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:10.658572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582654a9-c12f-4fc7-b061-c717ba4f78c0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:10.658921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582654a9-c12f-4fc7-b061-c717ba4f78c0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:10.659210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582654a9-c12f-4fc7-b061-c717ba4f78c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:10.685087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-582654a9-c12f-4fc7-b061-c717ba4f78c0 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:10.686710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-582654a9-c12f-4fc7-b061-c717ba4f78c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:10.714317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f3eb71-a03f-4978-b143-7b2baafa7c80 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:10.722463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675d27d-ac71-4973-a276-8e1c3d1b09f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:10.723205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675d27d-ac71-4973-a276-8e1c3d1b09f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:10.725330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675d27d-ac71-4973-a276-8e1c3d1b09f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:10.725330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675d27d-ac71-4973-a276-8e1c3d1b09f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:10.725330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675d27d-ac71-4973-a276-8e1c3d1b09f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:10.727379 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675d27d-ac71-4973-a276-8e1c3d1b09f4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:10.729239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:10.741217 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 805/1617] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:10.745097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675d27d-ac71-4973-a276-8e1c3d1b09f4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:10.754562 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675d27d-ac71-4973-a276-8e1c3d1b09f4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:10.763621 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6d619-f31d-4d8a-a747-962d502ebc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:10.764664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6d619-f31d-4d8a-a747-962d502ebc48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:10.766386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6d619-f31d-4d8a-a747-962d502ebc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:10.767284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef6d619-f31d-4d8a-a747-962d502ebc48 admin admin] PATH_INFO: `/v3/projects/bcd6fe4cc3554b888bb25819f5fac1e4/users/14e51ac136874806b4b903235697b404/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:10.791065 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 813/1618] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:10.791382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6d619-f31d-4d8a-a747-962d502ebc48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcd6fe4cc3554b888bb25819f5fac1e4, user_id=14e51ac136874806b4b903235697b404, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:10.792015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582654a9-c12f-4fc7-b061-c717ba4f78c0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:10.794468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef6d619-f31d-4d8a-a747-962d502ebc48 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:10.835542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:10.886043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 806/1619] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:10 2022] PUT /identity/v3/projects/bcd6fe4cc3554b888bb25819f5fac1e4/users/14e51ac136874806b4b903235697b404/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:10.888470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6d619-f31d-4d8a-a747-962d502ebc48 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:10.894249 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582654a9-c12f-4fc7-b061-c717ba4f78c0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:10.958734 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef6d619-f31d-4d8a-a747-962d502ebc48 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:10.963796 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3394d14-ca71-4327-996a-bb29c75c19a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:10.964458 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3394d14-ca71-4327-996a-bb29c75c19a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:10.964682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3394d14-ca71-4327-996a-bb29c75c19a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:10.964959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3394d14-ca71-4327-996a-bb29c75c19a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:10.965587 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3394d14-ca71-4327-996a-bb29c75c19a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:10.967527 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3394d14-ca71-4327-996a-bb29c75c19a4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:10.976846 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317babd-fb55-428e-97f9-9abc831f72a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:10.977682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317babd-fb55-428e-97f9-9abc831f72a6 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:10.978270 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317babd-fb55-428e-97f9-9abc831f72a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:10.978270 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d317babd-fb55-428e-97f9-9abc831f72a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:11.037763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d317babd-fb55-428e-97f9-9abc831f72a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:11.039574 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d317babd-fb55-428e-97f9-9abc831f72a6 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:11.049495 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 814/1620] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:11.051287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317babd-fb55-428e-97f9-9abc831f72a6 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:11.071331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d317babd-fb55-428e-97f9-9abc831f72a6 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:11.082703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6b91d-ca9b-4382-a1bb-10d1631b33e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:11.083510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6b91d-ca9b-4382-a1bb-10d1631b33e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:11.084051 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6b91d-ca9b-4382-a1bb-10d1631b33e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:11.084335 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e6b91d-ca9b-4382-a1bb-10d1631b33e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:11.085322 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6b91d-ca9b-4382-a1bb-10d1631b33e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:11.087856 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e6b91d-ca9b-4382-a1bb-10d1631b33e5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:11.099706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 807/1621] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:10 2022] POST /identity/v3/users => generated 292 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:11.099706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3394d14-ca71-4327-996a-bb29c75c19a4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:11.124673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3394d14-ca71-4327-996a-bb29c75c19a4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:11.132408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6bf64f-f917-4cce-bf33-ab9be6e0eaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:11.134805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bf64f-f917-4cce-bf33-ab9be6e0eaee admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:11.135224 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bf64f-f917-4cce-bf33-ab9be6e0eaee admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:11.136004 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6bf64f-f917-4cce-bf33-ab9be6e0eaee admin admin] PATH_INFO: `/v3/projects/e431d84366b642efb534265e38fa41a3/users/6360b36885b049f483d5979132b7e2de/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:11.146949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:11.157606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bf64f-f917-4cce-bf33-ab9be6e0eaee admin admin] RBAC: Authorizing `identity:create_grant(project_id=e431d84366b642efb534265e38fa41a3, user_id=6360b36885b049f483d5979132b7e2de, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:11.160376 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6bf64f-f917-4cce-bf33-ab9be6e0eaee admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:11.224774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 815/1622] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:11 2022] POST /identity/v3/users => generated 297 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:11.225866 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6b91d-ca9b-4382-a1bb-10d1631b33e5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:11.270877 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 808/1623] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:11 2022] PUT /identity/v3/projects/e431d84366b642efb534265e38fa41a3/users/6360b36885b049f483d5979132b7e2de/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:11.272827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db398da-3b85-4d10-9b97-5fed3afc3fc0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:11.273244 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db398da-3b85-4d10-9b97-5fed3afc3fc0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:11.273676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db398da-3b85-4d10-9b97-5fed3afc3fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:11.296666 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1db398da-3b85-4d10-9b97-5fed3afc3fc0 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:11.298014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1db398da-3b85-4d10-9b97-5fed3afc3fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:11.332871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e6b91d-ca9b-4382-a1bb-10d1631b33e5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:11.376701 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 809/1624] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:11.376701 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db398da-3b85-4d10-9b97-5fed3afc3fc0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:11.386400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db398da-3b85-4d10-9b97-5fed3afc3fc0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:11.392682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d6d30-bcf8-4603-bcac-d7afffdd2c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:11.394152 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d6d30-bcf8-4603-bcac-d7afffdd2c6e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:11.394443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d6d30-bcf8-4603-bcac-d7afffdd2c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:11.394706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1d6d30-bcf8-4603-bcac-d7afffdd2c6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:11.395356 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d6d30-bcf8-4603-bcac-d7afffdd2c6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:11.397053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1d6d30-bcf8-4603-bcac-d7afffdd2c6e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:11.407328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:11.413620 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 810/1625] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:11.414776 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d6d30-bcf8-4603-bcac-d7afffdd2c6e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:11.417308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acbd389-4802-42fa-a3b4-d3172d82c15e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:11.418298 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbd389-4802-42fa-a3b4-d3172d82c15e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:11.418799 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbd389-4802-42fa-a3b4-d3172d82c15e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:11.419316 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acbd389-4802-42fa-a3b4-d3172d82c15e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:11.427223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1d6d30-bcf8-4603-bcac-d7afffdd2c6e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:11.434280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56109028-2428-4d2e-95bc-6e519aa6fee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:11.438495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56109028-2428-4d2e-95bc-6e519aa6fee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:11.438495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56109028-2428-4d2e-95bc-6e519aa6fee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:11.438495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56109028-2428-4d2e-95bc-6e519aa6fee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:11.438495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56109028-2428-4d2e-95bc-6e519aa6fee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:11.438495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56109028-2428-4d2e-95bc-6e519aa6fee7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:11.440747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acbd389-4802-42fa-a3b4-d3172d82c15e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:11.442863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acbd389-4802-42fa-a3b4-d3172d82c15e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:11.457223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 811/1626] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:11.458524 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56109028-2428-4d2e-95bc-6e519aa6fee7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:11.474158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56109028-2428-4d2e-95bc-6e519aa6fee7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:11.481214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3256ce2-926a-41c3-98c1-09f2c8977309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:11.482110 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3256ce2-926a-41c3-98c1-09f2c8977309 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:11.482492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3256ce2-926a-41c3-98c1-09f2c8977309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:11.482866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3256ce2-926a-41c3-98c1-09f2c8977309 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:11.483933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3256ce2-926a-41c3-98c1-09f2c8977309 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:11.485711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3256ce2-926a-41c3-98c1-09f2c8977309 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:11.488814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 816/1627] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:11.490639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acbd389-4802-42fa-a3b4-d3172d82c15e service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:11.512064 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acbd389-4802-42fa-a3b4-d3172d82c15e service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:11.519833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 812/1628] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:11.520876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3256ce2-926a-41c3-98c1-09f2c8977309 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:11.524748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b38b4-fd25-4d5c-b762-aa40c718795a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:11.525560 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b38b4-fd25-4d5c-b762-aa40c718795a service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:11.525688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b38b4-fd25-4d5c-b762-aa40c718795a service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:11.525935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b38b4-fd25-4d5c-b762-aa40c718795a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:11.528980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:11.535167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3256ce2-926a-41c3-98c1-09f2c8977309 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:11.541774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d123f-0b09-453a-bb82-e300ca28390d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:11.542454 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d123f-0b09-453a-bb82-e300ca28390d admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:11.542744 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d123f-0b09-453a-bb82-e300ca28390d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:11.543021 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182d123f-0b09-453a-bb82-e300ca28390d admin admin] PATH_INFO: `/v3/projects/26bdd9d6e2e74909ae1030a4e845ac53/users/eb7dcd4c8d4d486bb28ce51a816ffe21/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:11.560616 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b38b4-fd25-4d5c-b762-aa40c718795a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:11.561382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d123f-0b09-453a-bb82-e300ca28390d admin admin] RBAC: Authorizing `identity:create_grant(project_id=26bdd9d6e2e74909ae1030a4e845ac53, user_id=eb7dcd4c8d4d486bb28ce51a816ffe21, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:11.562349 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b38b4-fd25-4d5c-b762-aa40c718795a service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:11.564042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182d123f-0b09-453a-bb82-e300ca28390d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:11.575133 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 817/1629] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:11.576392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b38b4-fd25-4d5c-b762-aa40c718795a service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:11.589753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b38b4-fd25-4d5c-b762-aa40c718795a service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:11.595201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0f088-467f-4735-8f22-6f54a75e8dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:11.596362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0f088-467f-4735-8f22-6f54a75e8dba admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:11.596833 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0f088-467f-4735-8f22-6f54a75e8dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:11.597310 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0f088-467f-4735-8f22-6f54a75e8dba admin admin] PATH_INFO: `/v3/projects/5bf22dee0e274298b7991d8d89241a8d/users/7c04a0b5622d4078bd088566e4a0fd9c/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:11.614588 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0f088-467f-4735-8f22-6f54a75e8dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf22dee0e274298b7991d8d89241a8d, user_id=7c04a0b5622d4078bd088566e4a0fd9c, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:11.618586 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0f088-467f-4735-8f22-6f54a75e8dba admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:11.698899 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 813/1630] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:11 2022] PUT /identity/v3/projects/26bdd9d6e2e74909ae1030a4e845ac53/users/eb7dcd4c8d4d486bb28ce51a816ffe21/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:11.699871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d123f-0b09-453a-bb82-e300ca28390d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:11.780217 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 818/1631] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:11 2022] PUT /identity/v3/projects/5bf22dee0e274298b7991d8d89241a8d/users/7c04a0b5622d4078bd088566e4a0fd9c/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:11.780217 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0f088-467f-4735-8f22-6f54a75e8dba admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:11.846864 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182d123f-0b09-453a-bb82-e300ca28390d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:11.926044 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0f088-467f-4735-8f22-6f54a75e8dba admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:11.933564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe687ec-9ae9-4883-bd1e-cb639513e782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:11.934559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe687ec-9ae9-4883-bd1e-cb639513e782 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:11.935092 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe687ec-9ae9-4883-bd1e-cb639513e782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:11.935633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe687ec-9ae9-4883-bd1e-cb639513e782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:11.936716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe687ec-9ae9-4883-bd1e-cb639513e782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:11.940081 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0699c4-6276-49af-9696-65439eb5a78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:11.940081 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0699c4-6276-49af-9696-65439eb5a78a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:11.940081 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0699c4-6276-49af-9696-65439eb5a78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:11.940422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0699c4-6276-49af-9696-65439eb5a78a admin admin] PATH_INFO: `/v3/projects/e431d84366b642efb534265e38fa41a3/users/6360b36885b049f483d5979132b7e2de/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:11.942038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe687ec-9ae9-4883-bd1e-cb639513e782 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:11.942749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0699c4-6276-49af-9696-65439eb5a78a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e431d84366b642efb534265e38fa41a3, user_id=6360b36885b049f483d5979132b7e2de, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:11.946706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0699c4-6276-49af-9696-65439eb5a78a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:11.966619 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 819/1632] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:11.970619 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe687ec-9ae9-4883-bd1e-cb639513e782 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:11.993269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe687ec-9ae9-4883-bd1e-cb639513e782 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:11.995189 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 814/1633] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:11 2022] PUT /identity/v3/projects/e431d84366b642efb534265e38fa41a3/users/6360b36885b049f483d5979132b7e2de/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:11.996702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a7164-1084-4b8f-84df-0f533025f53a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:11.996984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a7164-1084-4b8f-84df-0f533025f53a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:11.997262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a7164-1084-4b8f-84df-0f533025f53a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:11.998994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 815/1634] 10.222.0.67 () {54 vars in 961 bytes} [Mon Sep 26 18:03:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:03:12.000671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3e1dc-c480-4147-b267-2040b651b1fd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:12.000862 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3e1dc-c480-4147-b267-2040b651b1fd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:12.001051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c3e1dc-c480-4147-b267-2040b651b1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:12.053392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-85c3e1dc-c480-4147-b267-2040b651b1fd None None] MFA Rules not processed for user `ed9c5ada4eda4254a061acdfcc19f4e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:12.056735 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-85c3e1dc-c480-4147-b267-2040b651b1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:12.120025 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7d839-c44c-4608-84c3-5e6d665e53b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:12.120870 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7d839-c44c-4608-84c3-5e6d665e53b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:12.121802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7d839-c44c-4608-84c3-5e6d665e53b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:12.122181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7d839-c44c-4608-84c3-5e6d665e53b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:12.123501 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7d839-c44c-4608-84c3-5e6d665e53b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:12.125721 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c7d839-c44c-4608-84c3-5e6d665e53b7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:12.145462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 820/1635] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:12.145462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7d839-c44c-4608-84c3-5e6d665e53b7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:12.157706 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7d839-c44c-4608-84c3-5e6d665e53b7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:12.162754 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e50b47b-515e-4119-97d0-cbeda0d547b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:12.163474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50b47b-515e-4119-97d0-cbeda0d547b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:12.163792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50b47b-515e-4119-97d0-cbeda0d547b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:12.164862 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e50b47b-515e-4119-97d0-cbeda0d547b7 admin admin] PATH_INFO: `/v3/projects/26bdd9d6e2e74909ae1030a4e845ac53/users/eb7dcd4c8d4d486bb28ce51a816ffe21/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:12.166853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50b47b-515e-4119-97d0-cbeda0d547b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26bdd9d6e2e74909ae1030a4e845ac53, user_id=eb7dcd4c8d4d486bb28ce51a816ffe21, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:12.172756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 816/1636] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:11 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:12.172756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39467c1-521c-4bd8-87fa-035df9675399 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:12.173038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e50b47b-515e-4119-97d0-cbeda0d547b7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:12.173161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39467c1-521c-4bd8-87fa-035df9675399 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:12.173161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39467c1-521c-4bd8-87fa-035df9675399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:12.186019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 821/1637] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:12 2022] PUT /identity/v3/projects/26bdd9d6e2e74909ae1030a4e845ac53/users/eb7dcd4c8d4d486bb28ce51a816ffe21/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:12.188578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bde11-6211-4d0a-b7ff-b015a80db6c2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:12.204904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bde11-6211-4d0a-b7ff-b015a80db6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:12.204904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53bde11-6211-4d0a-b7ff-b015a80db6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:12.214740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:12.238799 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b39467c1-521c-4bd8-87fa-035df9675399 None None] MFA Rules not processed for user `6360b36885b049f483d5979132b7e2de`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:12.238799 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b39467c1-521c-4bd8-87fa-035df9675399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:12.244448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a53bde11-6211-4d0a-b7ff-b015a80db6c2 None None] MFA Rules not processed for user `b9ed29a3f6be4d64bb3751f9e95eb56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:12.245695 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a53bde11-6211-4d0a-b7ff-b015a80db6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:12.284669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:12.350748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 822/1638] 10.222.0.67 () {58 vars in 1058 bytes} [Mon Sep 26 18:03:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:12.351802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bde11-6211-4d0a-b7ff-b015a80db6c2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:12.374993 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 817/1639] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:12 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:12.375980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39467c1-521c-4bd8-87fa-035df9675399 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:12.438173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53bde11-6211-4d0a-b7ff-b015a80db6c2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:12.443828 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f907ac-d700-4077-ba09-70629050b3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:12.444788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f907ac-d700-4077-ba09-70629050b3f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:12.445081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f907ac-d700-4077-ba09-70629050b3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:12.446080 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f907ac-d700-4077-ba09-70629050b3f5 admin admin] PATH_INFO: `/v3/projects/5bf22dee0e274298b7991d8d89241a8d/users/7c04a0b5622d4078bd088566e4a0fd9c/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:12.447425 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39467c1-521c-4bd8-87fa-035df9675399 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:12.448582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f907ac-d700-4077-ba09-70629050b3f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bf22dee0e274298b7991d8d89241a8d, user_id=7c04a0b5622d4078bd088566e4a0fd9c, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:12.451319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f907ac-d700-4077-ba09-70629050b3f5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:12.452336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6015e7b-e04d-4b29-8915-de060a4f4653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:12.452969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6015e7b-e04d-4b29-8915-de060a4f4653 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:12.453245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6015e7b-e04d-4b29-8915-de060a4f4653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:12.453522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6015e7b-e04d-4b29-8915-de060a4f4653 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:12.454135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6015e7b-e04d-4b29-8915-de060a4f4653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:12.456043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6015e7b-e04d-4b29-8915-de060a4f4653 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:12.462649 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 823/1640] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:12 2022] PUT /identity/v3/projects/5bf22dee0e274298b7991d8d89241a8d/users/7c04a0b5622d4078bd088566e4a0fd9c/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:12.464154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33573e45-1642-4496-b9ec-0ca0dd3fe5e1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:12.464396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33573e45-1642-4496-b9ec-0ca0dd3fe5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:12.464496 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33573e45-1642-4496-b9ec-0ca0dd3fe5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:12.507367 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-33573e45-1642-4496-b9ec-0ca0dd3fe5e1 None None] MFA Rules not processed for user `eb7dcd4c8d4d486bb28ce51a816ffe21`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:12.509951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-33573e45-1642-4496-b9ec-0ca0dd3fe5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:12.569587 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 818/1641] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:12 2022] POST /identity/v3/projects => generated 405 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:12.572707 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadca63-0124-4d3d-bc6f-3e980f40ad90 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:12.573354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadca63-0124-4d3d-bc6f-3e980f40ad90 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:12.574027 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deadca63-0124-4d3d-bc6f-3e980f40ad90 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:12.577391 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 819/1642] 10.222.0.67 () {54 vars in 912 bytes} [Mon Sep 26 18:03:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:03:12.579154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadca63-0124-4d3d-bc6f-3e980f40ad90 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:12.601704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 824/1643] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:12.603110 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdae7a-8291-4d03-a400-8e3df1a04d42 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:12.603395 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdae7a-8291-4d03-a400-8e3df1a04d42 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:12.603690 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdae7a-8291-4d03-a400-8e3df1a04d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:12.641087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d9fdae7a-8291-4d03-a400-8e3df1a04d42 None None] MFA Rules not processed for user `7c04a0b5622d4078bd088566e4a0fd9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:12.642872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d9fdae7a-8291-4d03-a400-8e3df1a04d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:12.703000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deadca63-0124-4d3d-bc6f-3e980f40ad90 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:12.713468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8110ae5-47e1-4005-9fd7-f251dc847584 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:12.713468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8110ae5-47e1-4005-9fd7-f251dc847584 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:12.713468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8110ae5-47e1-4005-9fd7-f251dc847584 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:12.713468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8110ae5-47e1-4005-9fd7-f251dc847584 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:12.748212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 825/1644] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:12 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:12.749059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdae7a-8291-4d03-a400-8e3df1a04d42 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:12.801645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8110ae5-47e1-4005-9fd7-f251dc847584 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:12.803163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8110ae5-47e1-4005-9fd7-f251dc847584 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:12.816405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 820/1645] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:12 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:12.830049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f855f-52e2-4f04-8179-964628ce5ea2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:12.831806 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f855f-52e2-4f04-8179-964628ce5ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:12.832189 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246f855f-52e2-4f04-8179-964628ce5ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:12.834329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdae7a-8291-4d03-a400-8e3df1a04d42 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:12.839488 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c028e-9ca5-4def-b534-6abd6ca71bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:12.840239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c028e-9ca5-4def-b534-6abd6ca71bab admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:12.840590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c028e-9ca5-4def-b534-6abd6ca71bab admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:12.841070 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c028e-9ca5-4def-b534-6abd6ca71bab admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:12.842661 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c028e-9ca5-4def-b534-6abd6ca71bab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:12.844076 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51c028e-9ca5-4def-b534-6abd6ca71bab admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:12.855869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-246f855f-52e2-4f04-8179-964628ce5ea2 None None] MFA Rules not processed for user `b9ed29a3f6be4d64bb3751f9e95eb56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:12.857125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-246f855f-52e2-4f04-8179-964628ce5ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:12.960293 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 821/1646] 10.222.0.67 () {58 vars in 1009 bytes} [Mon Sep 26 18:03:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:12.960293 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f855f-52e2-4f04-8179-964628ce5ea2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:12.968633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246f855f-52e2-4f04-8179-964628ce5ea2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:12.972291 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7ebd1-818b-4070-9793-2ef625867f03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:12.972886 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7ebd1-818b-4070-9793-2ef625867f03 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:12.973166 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7ebd1-818b-4070-9793-2ef625867f03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:12.973467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac7ebd1-818b-4070-9793-2ef625867f03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:12.987716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 826/1647] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:12 2022] POST /identity/v3/users => generated 309 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:12.988617 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c028e-9ca5-4def-b534-6abd6ca71bab admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:13.000999 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c028e-9ca5-4def-b534-6abd6ca71bab admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:13.005139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bab30-9d39-49fe-8cb4-4620c71d3c3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:13.005802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bab30-9d39-49fe-8cb4-4620c71d3c3b service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:13.006024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bab30-9d39-49fe-8cb4-4620c71d3c3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:13.006301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9bab30-9d39-49fe-8cb4-4620c71d3c3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:13.016475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bab30-9d39-49fe-8cb4-4620c71d3c3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:13.017962 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9bab30-9d39-49fe-8cb4-4620c71d3c3b service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:13.029352 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 827/1648] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:12 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:13.045297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bab30-9d39-49fe-8cb4-4620c71d3c3b service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:13.071438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9bab30-9d39-49fe-8cb4-4620c71d3c3b service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:13.080632 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eef572-8b92-461f-b320-f102968035f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:13.083034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eef572-8b92-461f-b320-f102968035f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:13.083034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eef572-8b92-461f-b320-f102968035f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:13.083034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eef572-8b92-461f-b320-f102968035f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:13.083629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eef572-8b92-461f-b320-f102968035f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:13.084979 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac7ebd1-818b-4070-9793-2ef625867f03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:13.087534 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eef572-8b92-461f-b320-f102968035f8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:13.088285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac7ebd1-818b-4070-9793-2ef625867f03 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:13.100485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 822/1649] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:13.104935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 828/1650] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:13.127010 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7ebd1-818b-4070-9793-2ef625867f03 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:13.133415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac7ebd1-818b-4070-9793-2ef625867f03 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:13.163098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c25b50-954a-411b-8132-0bbdbf50ed49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:13.163843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c25b50-954a-411b-8132-0bbdbf50ed49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:13.164170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c25b50-954a-411b-8132-0bbdbf50ed49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:13.164589 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c25b50-954a-411b-8132-0bbdbf50ed49 admin admin] PATH_INFO: `/v3/projects/bcb1830b4d004072a2ae91a699f096bb/users/41492c51c9fc4c248be80a8fa0b063e5/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:13.589026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c25b50-954a-411b-8132-0bbdbf50ed49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb1830b4d004072a2ae91a699f096bb, user_id=41492c51c9fc4c248be80a8fa0b063e5, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:13.589026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c25b50-954a-411b-8132-0bbdbf50ed49 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:13.590451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eef572-8b92-461f-b320-f102968035f8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:13.590451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eef572-8b92-461f-b320-f102968035f8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:13.590451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fea8db-66c0-4446-94fe-2068d1e6f677 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:13.590451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fea8db-66c0-4446-94fe-2068d1e6f677 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:13.592844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fea8db-66c0-4446-94fe-2068d1e6f677 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:13.592844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fea8db-66c0-4446-94fe-2068d1e6f677 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:13.592844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fea8db-66c0-4446-94fe-2068d1e6f677 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:13.592844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fea8db-66c0-4446-94fe-2068d1e6f677 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:13.592844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 829/1651] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:13.595074 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:13.595074 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 823/1652] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:13 2022] PUT /identity/v3/projects/bcb1830b4d004072a2ae91a699f096bb/users/41492c51c9fc4c248be80a8fa0b063e5/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:13.597431 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fea8db-66c0-4446-94fe-2068d1e6f677 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:13.597431 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fea8db-66c0-4446-94fe-2068d1e6f677 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:13.598981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7a058-ebb2-4f2f-b769-b8e066dc74be None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:13.598981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7a058-ebb2-4f2f-b769-b8e066dc74be None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:13.598981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7a058-ebb2-4f2f-b769-b8e066dc74be None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:13.598981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 824/1653] 10.222.0.67 () {54 vars in 912 bytes} [Mon Sep 26 18:03:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 18:03:13.598981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f966cbe-29ae-44ce-b46f-03fb0a519bef None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:13.598981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f966cbe-29ae-44ce-b46f-03fb0a519bef None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:13.600075 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f966cbe-29ae-44ce-b46f-03fb0a519bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:13.600759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99b676-58e5-4073-9e21-deed686eb0cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:13.600759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99b676-58e5-4073-9e21-deed686eb0cf admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:13.601116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1f966cbe-29ae-44ce-b46f-03fb0a519bef None None] MFA Rules not processed for user `b9ed29a3f6be4d64bb3751f9e95eb56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:13.601177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99b676-58e5-4073-9e21-deed686eb0cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:13.601177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99b676-58e5-4073-9e21-deed686eb0cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:13.601177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99b676-58e5-4073-9e21-deed686eb0cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:13.602024 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1f966cbe-29ae-44ce-b46f-03fb0a519bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:13.602104 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99b676-58e5-4073-9e21-deed686eb0cf admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:13.606022 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 830/1654] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:13 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:13.625750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99b676-58e5-4073-9e21-deed686eb0cf admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:13.639227 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99b676-58e5-4073-9e21-deed686eb0cf admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:13.644039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e6f44-6446-4fc4-adc2-df279d960af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:13.644920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e6f44-6446-4fc4-adc2-df279d960af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:13.645494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e6f44-6446-4fc4-adc2-df279d960af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:13.645758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e6f44-6446-4fc4-adc2-df279d960af5 admin admin] PATH_INFO: `/v3/projects/bcb1830b4d004072a2ae91a699f096bb/users/41492c51c9fc4c248be80a8fa0b063e5/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:13.647970 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e6f44-6446-4fc4-adc2-df279d960af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcb1830b4d004072a2ae91a699f096bb, user_id=41492c51c9fc4c248be80a8fa0b063e5, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:13.651058 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e6f44-6446-4fc4-adc2-df279d960af5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:13.674877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 831/1655] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:13 2022] PUT /identity/v3/projects/bcb1830b4d004072a2ae91a699f096bb/users/41492c51c9fc4c248be80a8fa0b063e5/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:13.685305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 825/1656] 10.222.0.67 () {58 vars in 1009 bytes} [Mon Sep 26 18:03:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:13.710347 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e6f44-6446-4fc4-adc2-df279d960af5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:13.711930 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f966cbe-29ae-44ce-b46f-03fb0a519bef None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:13.722905 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f966cbe-29ae-44ce-b46f-03fb0a519bef None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:13.776251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e6f44-6446-4fc4-adc2-df279d960af5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:13.780084 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a212818-3df1-4bb4-bd0e-1e921c923a04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:13.780698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a212818-3df1-4bb4-bd0e-1e921c923a04 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:13.780983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a212818-3df1-4bb4-bd0e-1e921c923a04 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:13.781258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a212818-3df1-4bb4-bd0e-1e921c923a04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:13.792373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5f684-aadb-4967-ac45-ad6491b3e0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:13.793017 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5f684-aadb-4967-ac45-ad6491b3e0e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:13.793302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5f684-aadb-4967-ac45-ad6491b3e0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:13.793575 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5f684-aadb-4967-ac45-ad6491b3e0e4 admin admin] PATH_INFO: `/v3/users/d652e96acb0545e591baec8551ccae63` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:13.794704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5f684-aadb-4967-ac45-ad6491b3e0e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d652e96acb0545e591baec8551ccae63)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:13.795748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-37a5f684-aadb-4967-ac45-ad6491b3e0e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 18:03:13.795748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 26 18:03:13.795748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: Sep 26 18:03:13.796942 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a5f684-aadb-4967-ac45-ad6491b3e0e4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:13.840068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a212818-3df1-4bb4-bd0e-1e921c923a04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:13.841494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a212818-3df1-4bb4-bd0e-1e921c923a04 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:13.850860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 832/1657] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:13.924761 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a212818-3df1-4bb4-bd0e-1e921c923a04 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:13.933999 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a212818-3df1-4bb4-bd0e-1e921c923a04 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:13.938175 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47050fbc-3339-4596-9435-c9e4f591a73d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:13.938907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47050fbc-3339-4596-9435-c9e4f591a73d service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:13.939188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47050fbc-3339-4596-9435-c9e4f591a73d service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:13.939458 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47050fbc-3339-4596-9435-c9e4f591a73d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:13.991269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:14.052005 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47050fbc-3339-4596-9435-c9e4f591a73d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:14.053574 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47050fbc-3339-4596-9435-c9e4f591a73d service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:14.068981 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 833/1658] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:14.115203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47050fbc-3339-4596-9435-c9e4f591a73d service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:14.141953 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47050fbc-3339-4596-9435-c9e4f591a73d service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:14.149837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a10c62-4ea0-4d6d-8218-9b6b6b12b880 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:14.151367 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a10c62-4ea0-4d6d-8218-9b6b6b12b880 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:14.152306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a10c62-4ea0-4d6d-8218-9b6b6b12b880 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:14.152660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a10c62-4ea0-4d6d-8218-9b6b6b12b880 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:14.220564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-37a5f684-aadb-4967-ac45-ad6491b3e0e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd652e96acb0545e591baec8551ccae63'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:14.292843 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a10c62-4ea0-4d6d-8218-9b6b6b12b880 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:14.295601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a10c62-4ea0-4d6d-8218-9b6b6b12b880 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:14.312008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 834/1659] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:03:14.370580 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-37a5f684-aadb-4967-ac45-ad6491b3e0e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd652e96acb0545e591baec8551ccae63'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:14.371439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-37a5f684-aadb-4967-ac45-ad6491b3e0e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd652e96acb0545e591baec8551ccae63'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:14.372190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-37a5f684-aadb-4967-ac45-ad6491b3e0e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd652e96acb0545e591baec8551ccae63'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:14.407844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-37a5f684-aadb-4967-ac45-ad6491b3e0e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd652e96acb0545e591baec8551ccae63'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:14.439208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 826/1660] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:03:13 2022] DELETE /identity/v3/users/d652e96acb0545e591baec8551ccae63 => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:14.441676 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd636f-0c6a-49b2-8ed0-a7549ec2aab5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:14.442034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd636f-0c6a-49b2-8ed0-a7549ec2aab5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:14.442401 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd636f-0c6a-49b2-8ed0-a7549ec2aab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:14.468703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-03bd636f-0c6a-49b2-8ed0-a7549ec2aab5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:14.470297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-03bd636f-0c6a-49b2-8ed0-a7549ec2aab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:14.575246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 835/1661] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:14.697633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5f684-aadb-4967-ac45-ad6491b3e0e4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:14.701130 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd636f-0c6a-49b2-8ed0-a7549ec2aab5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:14.703282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-03bd636f-0c6a-49b2-8ed0-a7549ec2aab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:14.703349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-37a5f684-aadb-4967-ac45-ad6491b3e0e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:14.767544 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a5f684-aadb-4967-ac45-ad6491b3e0e4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:14.794549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33501671-80fe-4c78-89bc-fed8550ddd28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:14.797861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33501671-80fe-4c78-89bc-fed8550ddd28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:14.798174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33501671-80fe-4c78-89bc-fed8550ddd28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:14.801157 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33501671-80fe-4c78-89bc-fed8550ddd28 admin admin] PATH_INFO: `/v3/users/6999c1f707f14c89b1556cdf0f886b53` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:14.814079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33501671-80fe-4c78-89bc-fed8550ddd28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6999c1f707f14c89b1556cdf0f886b53)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:14.825956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33501671-80fe-4c78-89bc-fed8550ddd28 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:14.916371 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd636f-0c6a-49b2-8ed0-a7549ec2aab5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:14.924069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7da51-a4f1-4568-999a-f3e1facc9ab4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:14.925585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7da51-a4f1-4568-999a-f3e1facc9ab4 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:14.925874 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7da51-a4f1-4568-999a-f3e1facc9ab4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:14.926157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7da51-a4f1-4568-999a-f3e1facc9ab4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:14.968623 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7da51-a4f1-4568-999a-f3e1facc9ab4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:14.970158 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7da51-a4f1-4568-999a-f3e1facc9ab4 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:14.995686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 836/1662] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:15.286376 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-33501671-80fe-4c78-89bc-fed8550ddd28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6999c1f707f14c89b1556cdf0f886b53'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:15.417928 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-33501671-80fe-4c78-89bc-fed8550ddd28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6999c1f707f14c89b1556cdf0f886b53'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:15.424664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-33501671-80fe-4c78-89bc-fed8550ddd28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6999c1f707f14c89b1556cdf0f886b53'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:15.425853 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-33501671-80fe-4c78-89bc-fed8550ddd28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6999c1f707f14c89b1556cdf0f886b53'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:15.456564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-33501671-80fe-4c78-89bc-fed8550ddd28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6999c1f707f14c89b1556cdf0f886b53'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:15.487206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 827/1663] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:03:14 2022] DELETE /identity/v3/users/6999c1f707f14c89b1556cdf0f886b53 => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:15.492834 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ec9fa-8719-481e-a5c9-78392d40cf8a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:15.493053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ec9fa-8719-481e-a5c9-78392d40cf8a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:15.493874 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ec9fa-8719-481e-a5c9-78392d40cf8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:15.534635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-db8ec9fa-8719-481e-a5c9-78392d40cf8a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:15.536639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-db8ec9fa-8719-481e-a5c9-78392d40cf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:15.661241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 837/1664] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:15.707617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33501671-80fe-4c78-89bc-fed8550ddd28 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:15.709273 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-33501671-80fe-4c78-89bc-fed8550ddd28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:15.845814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33501671-80fe-4c78-89bc-fed8550ddd28 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:15.851078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb0813-43fd-4df7-a79f-5aedc77ce275 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:15.855996 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb0813-43fd-4df7-a79f-5aedc77ce275 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:15.860198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb0813-43fd-4df7-a79f-5aedc77ce275 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:15.860666 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb0813-43fd-4df7-a79f-5aedc77ce275 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:15.912203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb0813-43fd-4df7-a79f-5aedc77ce275 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:15.918776 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb0813-43fd-4df7-a79f-5aedc77ce275 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:15.959222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 828/1665] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:16.324900 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:16.435248 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ec9fa-8719-481e-a5c9-78392d40cf8a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:16.437806 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-db8ec9fa-8719-481e-a5c9-78392d40cf8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:16.495827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ec9fa-8719-481e-a5c9-78392d40cf8a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:16.533867 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:16.537186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:16.540467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:16.548876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b admin admin] PATH_INFO: `/v3/projects/2054a4e806234580bd7d4e4ac5e97924` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:16.551402 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2054a4e806234580bd7d4e4ac5e97924)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:16.552906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 26 18:03:16.552906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Sep 26 18:03:16.552906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: Sep 26 18:03:16.555629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:16.802616 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:16.879471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2054a4e806234580bd7d4e4ac5e97924', 'request_id': 'req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:16.919625 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2054a4e806234580bd7d4e4ac5e97924', 'request_id': 'req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:17.035141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2054a4e806234580bd7d4e4ac5e97924', 'request_id': 'req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:17.061975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b admin admin] The token cache is being invalidate because project 2054a4e806234580bd7d4e4ac5e97924 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:03:17.062845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:17.064367 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2ad562b2-f5ae-41c5-a761-e37ccd2be95b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:17.070626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 838/1666] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:03:16 2022] DELETE /identity/v3/projects/2054a4e806234580bd7d4e4ac5e97924 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:17.075260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb0813-43fd-4df7-a79f-5aedc77ce275 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:17.091631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb0813-43fd-4df7-a79f-5aedc77ce275 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:17.349663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc57b8f-972c-4f10-881a-e6553a3d4af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:17.352635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc57b8f-972c-4f10-881a-e6553a3d4af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:17.360344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc57b8f-972c-4f10-881a-e6553a3d4af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:17.360808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc57b8f-972c-4f10-881a-e6553a3d4af6 admin admin] PATH_INFO: `/v3/users/6360b36885b049f483d5979132b7e2de` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:17.363017 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc57b8f-972c-4f10-881a-e6553a3d4af6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6360b36885b049f483d5979132b7e2de)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:17.365253 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc57b8f-972c-4f10-881a-e6553a3d4af6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:17.765082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ecc57b8f-972c-4f10-881a-e6553a3d4af6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6360b36885b049f483d5979132b7e2de'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:17.882593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ecc57b8f-972c-4f10-881a-e6553a3d4af6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6360b36885b049f483d5979132b7e2de'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:17.887032 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ecc57b8f-972c-4f10-881a-e6553a3d4af6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6360b36885b049f483d5979132b7e2de'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:17.893796 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ecc57b8f-972c-4f10-881a-e6553a3d4af6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6360b36885b049f483d5979132b7e2de'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:17.903964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ecc57b8f-972c-4f10-881a-e6553a3d4af6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6360b36885b049f483d5979132b7e2de'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:17.938144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 829/1667] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:03:17 2022] DELETE /identity/v3/users/6360b36885b049f483d5979132b7e2de => generated 0 bytes in 856 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:18.321735 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:18.402508 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7e63f-1d01-4216-9429-6fcaf96d509c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:18.403487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7e63f-1d01-4216-9429-6fcaf96d509c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:18.404052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c7e63f-1d01-4216-9429-6fcaf96d509c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:18.445711 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c4c7e63f-1d01-4216-9429-6fcaf96d509c None None] MFA Rules not processed for user `41492c51c9fc4c248be80a8fa0b063e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:18.451877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c4c7e63f-1d01-4216-9429-6fcaf96d509c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:18.616978 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 839/1668] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:18 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:18.870039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:19.410273 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc57b8f-972c-4f10-881a-e6553a3d4af6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:19.412576 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ecc57b8f-972c-4f10-881a-e6553a3d4af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:19.596288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc57b8f-972c-4f10-881a-e6553a3d4af6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:19.603767 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1025ea3-8868-48a3-b2f7-361c274eaba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:19.607002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1025ea3-8868-48a3-b2f7-361c274eaba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:19.607002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1025ea3-8868-48a3-b2f7-361c274eaba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:19.607002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1025ea3-8868-48a3-b2f7-361c274eaba6 admin admin] PATH_INFO: `/v3/projects/247ce86198ec46aa9a566403ccefaa28` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:19.607002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1025ea3-8868-48a3-b2f7-361c274eaba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=247ce86198ec46aa9a566403ccefaa28)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:19.610079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1025ea3-8868-48a3-b2f7-361c274eaba6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:19.659069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:19.730451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7e63f-1d01-4216-9429-6fcaf96d509c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:19.731940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c4c7e63f-1d01-4216-9429-6fcaf96d509c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:19.767493 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c7e63f-1d01-4216-9429-6fcaf96d509c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:19.799263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d1025ea3-8868-48a3-b2f7-361c274eaba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '247ce86198ec46aa9a566403ccefaa28', 'request_id': 'req-d1025ea3-8868-48a3-b2f7-361c274eaba6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:19.823827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d1025ea3-8868-48a3-b2f7-361c274eaba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '247ce86198ec46aa9a566403ccefaa28', 'request_id': 'req-d1025ea3-8868-48a3-b2f7-361c274eaba6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:19.909644 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d1025ea3-8868-48a3-b2f7-361c274eaba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '247ce86198ec46aa9a566403ccefaa28', 'request_id': 'req-d1025ea3-8868-48a3-b2f7-361c274eaba6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:19.924805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d1025ea3-8868-48a3-b2f7-361c274eaba6 admin admin] The token cache is being invalidate because project 247ce86198ec46aa9a566403ccefaa28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:03:19.927790 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d1025ea3-8868-48a3-b2f7-361c274eaba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:19.930647 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d1025ea3-8868-48a3-b2f7-361c274eaba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:19.937062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 830/1669] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:03:19 2022] DELETE /identity/v3/projects/247ce86198ec46aa9a566403ccefaa28 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:19.940382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9069630-be11-4b35-ba9f-5bc13645c81a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:19.940382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9069630-be11-4b35-ba9f-5bc13645c81a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:19.940382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9069630-be11-4b35-ba9f-5bc13645c81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:19.967709 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf889c-61ee-457a-ae6d-4a1f115adc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:19.971343 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf889c-61ee-457a-ae6d-4a1f115adc21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:19.971880 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf889c-61ee-457a-ae6d-4a1f115adc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:19.972499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf889c-61ee-457a-ae6d-4a1f115adc21 admin admin] PATH_INFO: `/v3/projects/e431d84366b642efb534265e38fa41a3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:19.978535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf889c-61ee-457a-ae6d-4a1f115adc21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e431d84366b642efb534265e38fa41a3)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:19.983331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf889c-61ee-457a-ae6d-4a1f115adc21 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:20.033035 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c9069630-be11-4b35-ba9f-5bc13645c81a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:20.035198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c9069630-be11-4b35-ba9f-5bc13645c81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:20.101191 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 831/1670] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:20.107193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bf0a5-88b8-443f-b46a-3bf179e3a437 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:20.108875 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bf0a5-88b8-443f-b46a-3bf179e3a437 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:20.109725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bf0a5-88b8-443f-b46a-3bf179e3a437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:20.159276 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fd4bf0a5-88b8-443f-b46a-3bf179e3a437 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:20.165413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fd4bf0a5-88b8-443f-b46a-3bf179e3a437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:20.185799 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 832/1671] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:20.194669 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4bf0a5-88b8-443f-b46a-3bf179e3a437 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:20.219512 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fddf889c-61ee-457a-ae6d-4a1f115adc21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e431d84366b642efb534265e38fa41a3', 'request_id': 'req-fddf889c-61ee-457a-ae6d-4a1f115adc21'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:20.233890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fddf889c-61ee-457a-ae6d-4a1f115adc21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e431d84366b642efb534265e38fa41a3', 'request_id': 'req-fddf889c-61ee-457a-ae6d-4a1f115adc21'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:20.234914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4bf0a5-88b8-443f-b46a-3bf179e3a437 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:20.326927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615f768-0175-45bb-95a4-f5606832ff52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:20.326927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615f768-0175-45bb-95a4-f5606832ff52 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:20.326927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615f768-0175-45bb-95a4-f5606832ff52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:20.326927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615f768-0175-45bb-95a4-f5606832ff52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:20.327422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615f768-0175-45bb-95a4-f5606832ff52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:20.327422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615f768-0175-45bb-95a4-f5606832ff52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:20.327422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615f768-0175-45bb-95a4-f5606832ff52 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:20.352366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fddf889c-61ee-457a-ae6d-4a1f115adc21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e431d84366b642efb534265e38fa41a3', 'request_id': 'req-fddf889c-61ee-457a-ae6d-4a1f115adc21'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:20.360777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 833/1672] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:03:20.363729 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615f768-0175-45bb-95a4-f5606832ff52 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:20.380473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:20.387964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615f768-0175-45bb-95a4-f5606832ff52 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:20.397919 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fddf889c-61ee-457a-ae6d-4a1f115adc21 admin admin] The token cache is being invalidate because project e431d84366b642efb534265e38fa41a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:03:20.398675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fddf889c-61ee-457a-ae6d-4a1f115adc21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:20.401401 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fddf889c-61ee-457a-ae6d-4a1f115adc21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:20.402165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccd301-e01f-4ce2-9a90-c9de24443dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:20.402861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccd301-e01f-4ce2-9a90-c9de24443dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:20.407272 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccd301-e01f-4ce2-9a90-c9de24443dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:20.407272 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccd301-e01f-4ce2-9a90-c9de24443dc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:20.407272 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccd301-e01f-4ce2-9a90-c9de24443dc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:20.407540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 840/1673] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:03:19 2022] DELETE /identity/v3/projects/e431d84366b642efb534265e38fa41a3 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:20.410629 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccd301-e01f-4ce2-9a90-c9de24443dc8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:20.411269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef78a1-6fe8-40b2-b781-357246002eb7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:20.412368 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef78a1-6fe8-40b2-b781-357246002eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:20.412792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef78a1-6fe8-40b2-b781-357246002eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:20.466805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-82ef78a1-6fe8-40b2-b781-357246002eb7 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:20.468131 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-82ef78a1-6fe8-40b2-b781-357246002eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:20.480887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 841/1674] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:20.489643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391562da-d637-477c-975d-5908cf36198b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:20.489977 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391562da-d637-477c-975d-5908cf36198b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:20.490259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391562da-d637-477c-975d-5908cf36198b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:20.524374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-391562da-d637-477c-975d-5908cf36198b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:20.526054 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-391562da-d637-477c-975d-5908cf36198b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:20.553022 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 842/1675] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:20.555352 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391562da-d637-477c-975d-5908cf36198b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:20.572932 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391562da-d637-477c-975d-5908cf36198b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:20.579431 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc090c-3b9a-4226-9dd0-82dffcfffeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:20.580181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc090c-3b9a-4226-9dd0-82dffcfffeeb admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:20.580463 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc090c-3b9a-4226-9dd0-82dffcfffeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:20.580734 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc090c-3b9a-4226-9dd0-82dffcfffeeb admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:20.581513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc090c-3b9a-4226-9dd0-82dffcfffeeb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:20.581817 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc090c-3b9a-4226-9dd0-82dffcfffeeb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:20.583685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc090c-3b9a-4226-9dd0-82dffcfffeeb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:20.602613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 843/1677] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:03:20.608559 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 834/1677] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:20 2022] POST /identity/v3/projects => generated 377 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:20.608686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc090c-3b9a-4226-9dd0-82dffcfffeeb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:20.608783 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccd301-e01f-4ce2-9a90-c9de24443dc8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:20.608854 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ebfc090c-3b9a-4226-9dd0-82dffcfffeeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:20.622061 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc090c-3b9a-4226-9dd0-82dffcfffeeb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:20.755270 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec412d6-3eb3-47ca-8b9e-c62467e6dd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:20.757689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec412d6-3eb3-47ca-8b9e-c62467e6dd69 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:20.758126 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec412d6-3eb3-47ca-8b9e-c62467e6dd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:20.771525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec412d6-3eb3-47ca-8b9e-c62467e6dd69 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:20.771525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec412d6-3eb3-47ca-8b9e-c62467e6dd69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:20.771905 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccd301-e01f-4ce2-9a90-c9de24443dc8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:20.771979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec412d6-3eb3-47ca-8b9e-c62467e6dd69 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:20.777139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859eb4e-e00d-4b7c-bc40-d8d4ab04fbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:20.777967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859eb4e-e00d-4b7c-bc40-d8d4ab04fbf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:20.778906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859eb4e-e00d-4b7c-bc40-d8d4ab04fbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:20.779118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859eb4e-e00d-4b7c-bc40-d8d4ab04fbf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:20.779814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859eb4e-e00d-4b7c-bc40-d8d4ab04fbf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:20.786003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859eb4e-e00d-4b7c-bc40-d8d4ab04fbf0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:20.901997 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:20.909226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 844/1678] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:20 2022] POST /identity/v3/users => generated 289 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:20.910813 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d1a0c-2923-4546-a129-09447a7c129a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:20.911206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d1a0c-2923-4546-a129-09447a7c129a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:20.911572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3d1a0c-2923-4546-a129-09447a7c129a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:20.914190 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 845/1679] 10.222.0.67 () {54 vars in 921 bytes} [Mon Sep 26 18:03:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:03:20.915450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d1a0c-2923-4546-a129-09447a7c129a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:20.935216 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3d1a0c-2923-4546-a129-09447a7c129a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:20.941815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b423e-7591-4aed-bd6e-652cc82b6ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:20.943461 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b423e-7591-4aed-bd6e-652cc82b6ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:20.943904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b423e-7591-4aed-bd6e-652cc82b6ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:20.944276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b423e-7591-4aed-bd6e-652cc82b6ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:20.947454 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b423e-7591-4aed-bd6e-652cc82b6ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:20.950425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b423e-7591-4aed-bd6e-652cc82b6ba2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:20.981728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 835/1680] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:20 2022] POST /identity/v3/projects => generated 367 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:21.000246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab4282-04c3-41d6-8a7a-850543b1e91c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:21.000246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab4282-04c3-41d6-8a7a-850543b1e91c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:21.000246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab4282-04c3-41d6-8a7a-850543b1e91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:21.001174 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 846/1681] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:21.001174 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b423e-7591-4aed-bd6e-652cc82b6ba2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:21.030172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b423e-7591-4aed-bd6e-652cc82b6ba2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:21.040201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6bab4282-04c3-41d6-8a7a-850543b1e91c None None] MFA Rules not processed for user `22a512e0a64e420abf15b2150ccc2b32`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:21.042824 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6bab4282-04c3-41d6-8a7a-850543b1e91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:21.186247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b792c-b05e-4257-b7bd-ddf68b9c86ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:21.187010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b792c-b05e-4257-b7bd-ddf68b9c86ef admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:21.188839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b792c-b05e-4257-b7bd-ddf68b9c86ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:21.188839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b792c-b05e-4257-b7bd-ddf68b9c86ef admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:21.189451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b792c-b05e-4257-b7bd-ddf68b9c86ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:21.192241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b792c-b05e-4257-b7bd-ddf68b9c86ef admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:21.210480 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 836/1682] 10.222.0.67 () {58 vars in 1018 bytes} [Mon Sep 26 18:03:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:21.212355 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab4282-04c3-41d6-8a7a-850543b1e91c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:21.226005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab4282-04c3-41d6-8a7a-850543b1e91c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:21.233975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbad14-6866-4918-80d6-802cd2daae7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:21.235578 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbad14-6866-4918-80d6-802cd2daae7c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:21.235578 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbad14-6866-4918-80d6-802cd2daae7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:21.235578 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdbad14-6866-4918-80d6-802cd2daae7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:21.235935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbad14-6866-4918-80d6-802cd2daae7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:21.239201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdbad14-6866-4918-80d6-802cd2daae7c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:21.266949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 837/1683] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:21.268193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbad14-6866-4918-80d6-802cd2daae7c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:21.270686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4fdbad14-6866-4918-80d6-802cd2daae7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:21.293911 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdbad14-6866-4918-80d6-802cd2daae7c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:21.316387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 847/1684] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:20 2022] POST /identity/v3/users => generated 284 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:21.317488 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b792c-b05e-4257-b7bd-ddf68b9c86ef admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:21.328318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e5ec2-a0a0-484c-a7c7-118d3f15752a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:21.328992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e5ec2-a0a0-484c-a7c7-118d3f15752a service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:21.329375 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e5ec2-a0a0-484c-a7c7-118d3f15752a service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:21.329765 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e5ec2-a0a0-484c-a7c7-118d3f15752a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:21.334184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b792c-b05e-4257-b7bd-ddf68b9c86ef admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:21.352079 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b011d3d-bcb7-40bc-a216-c5c14140a27a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:21.357971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b011d3d-bcb7-40bc-a216-c5c14140a27a admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:21.357971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b011d3d-bcb7-40bc-a216-c5c14140a27a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:21.357971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b011d3d-bcb7-40bc-a216-c5c14140a27a admin admin] PATH_INFO: `/v3/projects/2d51270a9feb460599c2b76b3774546b/users/52567607814b4dce8b35cda56df6ad58/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:21.368212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e5ec2-a0a0-484c-a7c7-118d3f15752a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:21.370136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e5ec2-a0a0-484c-a7c7-118d3f15752a service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:21.399594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b011d3d-bcb7-40bc-a216-c5c14140a27a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d51270a9feb460599c2b76b3774546b, user_id=52567607814b4dce8b35cda56df6ad58, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:21.402931 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b011d3d-bcb7-40bc-a216-c5c14140a27a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:21.413190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 838/1685] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:21.415341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e5ec2-a0a0-484c-a7c7-118d3f15752a service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:21.434338 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e5ec2-a0a0-484c-a7c7-118d3f15752a service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:21.441564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78e197-c99d-4183-a3eb-e46be9c35251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:21.442474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78e197-c99d-4183-a3eb-e46be9c35251 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:21.442948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78e197-c99d-4183-a3eb-e46be9c35251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:21.444559 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78e197-c99d-4183-a3eb-e46be9c35251 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:21.449886 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78e197-c99d-4183-a3eb-e46be9c35251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:21.452727 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78e197-c99d-4183-a3eb-e46be9c35251 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:21.508043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 839/1686] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:21.518885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78e197-c99d-4183-a3eb-e46be9c35251 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:21.550457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78e197-c99d-4183-a3eb-e46be9c35251 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:21.555641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868fbcb-d936-4240-abe8-b616547cf2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:21.556938 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868fbcb-d936-4240-abe8-b616547cf2e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:21.557473 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868fbcb-d936-4240-abe8-b616547cf2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:21.557982 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868fbcb-d936-4240-abe8-b616547cf2e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:21.558852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868fbcb-d936-4240-abe8-b616547cf2e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:21.561430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868fbcb-d936-4240-abe8-b616547cf2e9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:21.579672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 848/1687] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:21 2022] PUT /identity/v3/projects/2d51270a9feb460599c2b76b3774546b/users/52567607814b4dce8b35cda56df6ad58/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:21.592642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dd363-b171-481c-a33c-9427306e8c41 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:21.592948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dd363-b171-481c-a33c-9427306e8c41 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:21.593210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55dd363-b171-481c-a33c-9427306e8c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:21.598661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 840/1688] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:21.600493 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868fbcb-d936-4240-abe8-b616547cf2e9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:21.612059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868fbcb-d936-4240-abe8-b616547cf2e9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:21.649009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a55dd363-b171-481c-a33c-9427306e8c41 None None] MFA Rules not processed for user `52567607814b4dce8b35cda56df6ad58`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:21.652139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a55dd363-b171-481c-a33c-9427306e8c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:21.755468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b7908-b8c5-4f42-8c30-cacb201998f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:21.756219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b7908-b8c5-4f42-8c30-cacb201998f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:21.756504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b7908-b8c5-4f42-8c30-cacb201998f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:21.756786 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b7908-b8c5-4f42-8c30-cacb201998f0 admin admin] PATH_INFO: `/v3/projects/69f4982ba2e5443d9587c84439068bb0/users/8a6b49e8387e458ba5e5dd40575479df/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:21.819387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b7908-b8c5-4f42-8c30-cacb201998f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69f4982ba2e5443d9587c84439068bb0, user_id=8a6b49e8387e458ba5e5dd40575479df, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:21.823995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01b7908-b8c5-4f42-8c30-cacb201998f0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:21.871327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 849/1689] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:21.878177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dd363-b171-481c-a33c-9427306e8c41 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:21.936843 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55dd363-b171-481c-a33c-9427306e8c41 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:21.937095 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 841/1690] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:21 2022] PUT /identity/v3/projects/69f4982ba2e5443d9587c84439068bb0/users/8a6b49e8387e458ba5e5dd40575479df/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:21.949701 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594e1ed-a523-4ac9-80a1-0e70653032fc None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:21.961346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594e1ed-a523-4ac9-80a1-0e70653032fc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:21.961346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b594e1ed-a523-4ac9-80a1-0e70653032fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:22.033772 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b594e1ed-a523-4ac9-80a1-0e70653032fc None None] MFA Rules not processed for user `8a6b49e8387e458ba5e5dd40575479df`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:22.035055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b594e1ed-a523-4ac9-80a1-0e70653032fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:22.094771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabfaee-421b-4f0d-88a3-e663a4c4f6a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:22.095407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabfaee-421b-4f0d-88a3-e663a4c4f6a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:22.095710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabfaee-421b-4f0d-88a3-e663a4c4f6a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:22.095982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabfaee-421b-4f0d-88a3-e663a4c4f6a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:22.102530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabfaee-421b-4f0d-88a3-e663a4c4f6a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:22.107253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabfaee-421b-4f0d-88a3-e663a4c4f6a8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:22.171762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 842/1691] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:22.181222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594e1ed-a523-4ac9-80a1-0e70653032fc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:22.207982 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b594e1ed-a523-4ac9-80a1-0e70653032fc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:22.223850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28ee9a-6a06-4f76-8e06-14fdf29819f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:22.224564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28ee9a-6a06-4f76-8e06-14fdf29819f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:22.224848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28ee9a-6a06-4f76-8e06-14fdf29819f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:22.225125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd28ee9a-6a06-4f76-8e06-14fdf29819f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:22.225800 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28ee9a-6a06-4f76-8e06-14fdf29819f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:22.231370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd28ee9a-6a06-4f76-8e06-14fdf29819f6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:22.288241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 850/1692] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:21 2022] POST /identity/v3/projects => generated 375 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:22.292917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabfaee-421b-4f0d-88a3-e663a4c4f6a8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:22.400841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 843/1693] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:22 2022] POST /identity/v3/projects => generated 367 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:22.402070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28ee9a-6a06-4f76-8e06-14fdf29819f6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:22.415091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabfaee-421b-4f0d-88a3-e663a4c4f6a8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:22.511057 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ada6e1-001c-4ce4-9b0c-2c8c07986319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:22.517669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ada6e1-001c-4ce4-9b0c-2c8c07986319 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:22.518834 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ada6e1-001c-4ce4-9b0c-2c8c07986319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:22.519714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ada6e1-001c-4ce4-9b0c-2c8c07986319 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:22.521426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ada6e1-001c-4ce4-9b0c-2c8c07986319 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:22.528909 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ada6e1-001c-4ce4-9b0c-2c8c07986319 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:22.566898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd28ee9a-6a06-4f76-8e06-14fdf29819f6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:22.577337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585cc17-cee3-4b4b-89d4-8e6661c162df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:22.578232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585cc17-cee3-4b4b-89d4-8e6661c162df admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:22.578664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585cc17-cee3-4b4b-89d4-8e6661c162df admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:22.579084 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0585cc17-cee3-4b4b-89d4-8e6661c162df admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:22.581531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585cc17-cee3-4b4b-89d4-8e6661c162df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:22.584707 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0585cc17-cee3-4b4b-89d4-8e6661c162df admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:22.627578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 851/1694] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:22 2022] POST /identity/v3/users => generated 294 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:22.636739 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ada6e1-001c-4ce4-9b0c-2c8c07986319 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:22.652807 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ada6e1-001c-4ce4-9b0c-2c8c07986319 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:22.662234 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53342a39-b6b9-4b3a-a4e2-6adb61c1a4e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:22.664373 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53342a39-b6b9-4b3a-a4e2-6adb61c1a4e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:22.681793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53342a39-b6b9-4b3a-a4e2-6adb61c1a4e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:22.681793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53342a39-b6b9-4b3a-a4e2-6adb61c1a4e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:22.681793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53342a39-b6b9-4b3a-a4e2-6adb61c1a4e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:22.681793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53342a39-b6b9-4b3a-a4e2-6adb61c1a4e9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:22.694705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 852/1695] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:22.699157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53342a39-b6b9-4b3a-a4e2-6adb61c1a4e9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:22.717318 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53342a39-b6b9-4b3a-a4e2-6adb61c1a4e9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:22.722402 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60debf01-9f45-4d40-a714-ce2ebb1c3b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:22.723515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60debf01-9f45-4d40-a714-ce2ebb1c3b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:22.724192 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60debf01-9f45-4d40-a714-ce2ebb1c3b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:22.724477 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60debf01-9f45-4d40-a714-ce2ebb1c3b97 admin admin] PATH_INFO: `/v3/projects/b29973e19d714c77b5c1da229b056cca/users/e307c03e8057433ab2be0773897c375b/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:22.747892 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60debf01-9f45-4d40-a714-ce2ebb1c3b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b29973e19d714c77b5c1da229b056cca, user_id=e307c03e8057433ab2be0773897c375b, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:22.756156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60debf01-9f45-4d40-a714-ce2ebb1c3b97 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:22.844682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 844/1696] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:22 2022] POST /identity/v3/users => generated 290 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:22.851455 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585cc17-cee3-4b4b-89d4-8e6661c162df admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:22.872937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0585cc17-cee3-4b4b-89d4-8e6661c162df admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:22.894197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa3f06-5e24-4c54-9a1b-d37f718c46c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:22.894999 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa3f06-5e24-4c54-9a1b-d37f718c46c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:22.895400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa3f06-5e24-4c54-9a1b-d37f718c46c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:22.895997 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aa3f06-5e24-4c54-9a1b-d37f718c46c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:22.896822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa3f06-5e24-4c54-9a1b-d37f718c46c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:22.898688 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aa3f06-5e24-4c54-9a1b-d37f718c46c8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:22.927366 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 845/1697] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:22.932077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa3f06-5e24-4c54-9a1b-d37f718c46c8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:22.949185 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 853/1698] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:22 2022] PUT /identity/v3/projects/b29973e19d714c77b5c1da229b056cca/users/e307c03e8057433ab2be0773897c375b/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:22.953916 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60debf01-9f45-4d40-a714-ce2ebb1c3b97 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:22.956173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aa3f06-5e24-4c54-9a1b-d37f718c46c8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:23.088709 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60debf01-9f45-4d40-a714-ce2ebb1c3b97 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:23.090124 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c049b7-20e8-4e00-8e08-3235e1baa1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:23.090989 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c049b7-20e8-4e00-8e08-3235e1baa1c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:23.091319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c049b7-20e8-4e00-8e08-3235e1baa1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:23.091602 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c049b7-20e8-4e00-8e08-3235e1baa1c5 admin admin] PATH_INFO: `/v3/projects/357d06436ed8423a8d49de78caecf243/users/333444f04b5c4a5bbafe075ccc0ebcc9/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:23.094430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5638e16-aa32-446e-8776-3cfbb55b1759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:23.095112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5638e16-aa32-446e-8776-3cfbb55b1759 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:23.095397 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5638e16-aa32-446e-8776-3cfbb55b1759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:23.095698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5638e16-aa32-446e-8776-3cfbb55b1759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:23.096583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5638e16-aa32-446e-8776-3cfbb55b1759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:23.098529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5638e16-aa32-446e-8776-3cfbb55b1759 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:23.116115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 854/1699] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:22 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:23.121138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5638e16-aa32-446e-8776-3cfbb55b1759 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:23.135250 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5638e16-aa32-446e-8776-3cfbb55b1759 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:23.147583 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c049b7-20e8-4e00-8e08-3235e1baa1c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=357d06436ed8423a8d49de78caecf243, user_id=333444f04b5c4a5bbafe075ccc0ebcc9, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:23.150868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c049b7-20e8-4e00-8e08-3235e1baa1c5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:23.168370 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63b6ef-e9ad-49a9-9918-2de39960e457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:23.169399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63b6ef-e9ad-49a9-9918-2de39960e457 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:23.169859 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63b6ef-e9ad-49a9-9918-2de39960e457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:23.170187 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63b6ef-e9ad-49a9-9918-2de39960e457 admin admin] PATH_INFO: `/v3/projects/b29973e19d714c77b5c1da229b056cca/users/e307c03e8057433ab2be0773897c375b/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:23.177255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63b6ef-e9ad-49a9-9918-2de39960e457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b29973e19d714c77b5c1da229b056cca, user_id=e307c03e8057433ab2be0773897c375b, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:23.180897 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d63b6ef-e9ad-49a9-9918-2de39960e457 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:23.227678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 855/1700] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:23 2022] PUT /identity/v3/projects/b29973e19d714c77b5c1da229b056cca/users/e307c03e8057433ab2be0773897c375b/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:23.229321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d038a88-806d-4700-88c9-04aba9a8e90f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:23.229614 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d038a88-806d-4700-88c9-04aba9a8e90f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:23.229882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d038a88-806d-4700-88c9-04aba9a8e90f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:23.299231 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 846/1701] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:22 2022] PUT /identity/v3/projects/357d06436ed8423a8d49de78caecf243/users/333444f04b5c4a5bbafe075ccc0ebcc9/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:23.300323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c049b7-20e8-4e00-8e08-3235e1baa1c5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:23.319396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3d038a88-806d-4700-88c9-04aba9a8e90f None None] MFA Rules not processed for user `e307c03e8057433ab2be0773897c375b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:23.322777 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3d038a88-806d-4700-88c9-04aba9a8e90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:23.460268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c049b7-20e8-4e00-8e08-3235e1baa1c5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:23.465372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 856/1702] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:23 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:23.477736 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1395b-fac4-4255-ba08-f0ce75dbc00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:23.478576 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1395b-fac4-4255-ba08-f0ce75dbc00c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:23.478870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1395b-fac4-4255-ba08-f0ce75dbc00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:23.479151 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a1395b-fac4-4255-ba08-f0ce75dbc00c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:23.479865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1395b-fac4-4255-ba08-f0ce75dbc00c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:23.481537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a1395b-fac4-4255-ba08-f0ce75dbc00c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:23.484726 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=5,keystone:SELECT=88,keystone:INSERT=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:23.487201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d038a88-806d-4700-88c9-04aba9a8e90f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:23.488732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3d038a88-806d-4700-88c9-04aba9a8e90f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:23.518721 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 847/1703] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:23 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:23.522750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1395b-fac4-4255-ba08-f0ce75dbc00c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:23.549933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a1395b-fac4-4255-ba08-f0ce75dbc00c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:23.560101 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6baf3b-38f9-4d4f-a025-bbc4b5f8cb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:23.560809 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6baf3b-38f9-4d4f-a025-bbc4b5f8cb3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:23.561095 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6baf3b-38f9-4d4f-a025-bbc4b5f8cb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:23.561375 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6baf3b-38f9-4d4f-a025-bbc4b5f8cb3a admin admin] PATH_INFO: `/v3/projects/357d06436ed8423a8d49de78caecf243/users/333444f04b5c4a5bbafe075ccc0ebcc9/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:23.576544 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6baf3b-38f9-4d4f-a025-bbc4b5f8cb3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=357d06436ed8423a8d49de78caecf243, user_id=333444f04b5c4a5bbafe075ccc0ebcc9, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:23.579470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6baf3b-38f9-4d4f-a025-bbc4b5f8cb3a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:23.601636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 848/1704] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:23 2022] PUT /identity/v3/projects/357d06436ed8423a8d49de78caecf243/users/333444f04b5c4a5bbafe075ccc0ebcc9/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:23.614513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1386a7-cd4c-45c0-8a68-8f9c10791207 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:23.615377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1386a7-cd4c-45c0-8a68-8f9c10791207 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:23.616458 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1386a7-cd4c-45c0-8a68-8f9c10791207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:23.658860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d038a88-806d-4700-88c9-04aba9a8e90f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:23.703679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2a1386a7-cd4c-45c0-8a68-8f9c10791207 None None] MFA Rules not processed for user `333444f04b5c4a5bbafe075ccc0ebcc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:23.705270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2a1386a7-cd4c-45c0-8a68-8f9c10791207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:23.760075 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be10a4e5-9641-43ff-9bb4-dccab6285dfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:23.761698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10a4e5-9641-43ff-9bb4-dccab6285dfa service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:23.762014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10a4e5-9641-43ff-9bb4-dccab6285dfa service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:23.762289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be10a4e5-9641-43ff-9bb4-dccab6285dfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:23.910978 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10a4e5-9641-43ff-9bb4-dccab6285dfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:23.917021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be10a4e5-9641-43ff-9bb4-dccab6285dfa service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:23.918408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 849/1705] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:23 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:23.924669 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21fa35-50ec-4f29-8513-5bd8a9e6b9aa None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:23.924669 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21fa35-50ec-4f29-8513-5bd8a9e6b9aa None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:23.924669 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd21fa35-50ec-4f29-8513-5bd8a9e6b9aa None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:23.926861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 850/1706] 10.222.0.67 () {54 vars in 921 bytes} [Mon Sep 26 18:03:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:03:23.929633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21fa35-50ec-4f29-8513-5bd8a9e6b9aa None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:23.933091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bd21fa35-50ec-4f29-8513-5bd8a9e6b9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:23.968670 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd21fa35-50ec-4f29-8513-5bd8a9e6b9aa None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:23.970308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 857/1707] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:23 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:23.993062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d2efa-7bb5-416c-b9e4-8d7adc2d5b61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:24.005378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:24.005378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d2efa-7bb5-416c-b9e4-8d7adc2d5b61 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:24.005378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d2efa-7bb5-416c-b9e4-8d7adc2d5b61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:24.005378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d2efa-7bb5-416c-b9e4-8d7adc2d5b61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:24.013030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d2efa-7bb5-416c-b9e4-8d7adc2d5b61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:24.016336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d2efa-7bb5-416c-b9e4-8d7adc2d5b61 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:24.046239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 851/1708] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:03:24.253640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee2cb0-45e5-4157-9633-0341cae44509 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:24.254264 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee2cb0-45e5-4157-9633-0341cae44509 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:24.258037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ee2cb0-45e5-4157-9633-0341cae44509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:24.326895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-34ee2cb0-45e5-4157-9633-0341cae44509 None None] MFA Rules not processed for user `22a512e0a64e420abf15b2150ccc2b32`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:24.326895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-34ee2cb0-45e5-4157-9633-0341cae44509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:24.327172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bab750-19f2-4f9d-8d19-d4eb31a89c0f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:24.327172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bab750-19f2-4f9d-8d19-d4eb31a89c0f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:24.327172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bab750-19f2-4f9d-8d19-d4eb31a89c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:24.342268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-60bab750-19f2-4f9d-8d19-d4eb31a89c0f None None] MFA Rules not processed for user `0bede3a601f941e7abc9fd16983679b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:24.344229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-60bab750-19f2-4f9d-8d19-d4eb31a89c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:24.402492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 858/1709] 10.222.0.67 () {58 vars in 1018 bytes} [Mon Sep 26 18:03:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:24.438814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee2cb0-45e5-4157-9633-0341cae44509 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:24.440709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 852/1710] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:24.460066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ee2cb0-45e5-4157-9633-0341cae44509 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:24.464285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bab750-19f2-4f9d-8d19-d4eb31a89c0f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:24.470819 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-60bab750-19f2-4f9d-8d19-d4eb31a89c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:24.480888 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08737ec8-15b4-4272-9693-a7e6034a8eda service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:24.482091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08737ec8-15b4-4272-9693-a7e6034a8eda service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:24.482091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08737ec8-15b4-4272-9693-a7e6034a8eda service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:24.482758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08737ec8-15b4-4272-9693-a7e6034a8eda service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:24.511724 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08737ec8-15b4-4272-9693-a7e6034a8eda service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:24.513787 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08737ec8-15b4-4272-9693-a7e6034a8eda service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:24.530273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 859/1711] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:24.616443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bab750-19f2-4f9d-8d19-d4eb31a89c0f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:24.627974 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16221b-3c1c-4d0c-9620-2587a4302aaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:24.628622 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16221b-3c1c-4d0c-9620-2587a4302aaf service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:24.628911 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16221b-3c1c-4d0c-9620-2587a4302aaf service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:24.629187 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16221b-3c1c-4d0c-9620-2587a4302aaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:24.641728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16221b-3c1c-4d0c-9620-2587a4302aaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:24.643971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16221b-3c1c-4d0c-9620-2587a4302aaf service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:24.680522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 853/1712] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:24.778835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08737ec8-15b4-4272-9693-a7e6034a8eda service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:24.780361 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-08737ec8-15b4-4272-9693-a7e6034a8eda service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:24.924724 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08737ec8-15b4-4272-9693-a7e6034a8eda service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:24.938552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70921733-2243-41d5-b714-dffa3c331c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:24.939686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70921733-2243-41d5-b714-dffa3c331c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:24.940157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70921733-2243-41d5-b714-dffa3c331c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:24.941413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70921733-2243-41d5-b714-dffa3c331c90 admin admin] PATH_INFO: `/v3/users/ed9c5ada4eda4254a061acdfcc19f4e1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:24.949013 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70921733-2243-41d5-b714-dffa3c331c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed9c5ada4eda4254a061acdfcc19f4e1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:24.952048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70921733-2243-41d5-b714-dffa3c331c90 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:25.274772 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-70921733-2243-41d5-b714-dffa3c331c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed9c5ada4eda4254a061acdfcc19f4e1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:25.369697 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-70921733-2243-41d5-b714-dffa3c331c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed9c5ada4eda4254a061acdfcc19f4e1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:25.370599 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-70921733-2243-41d5-b714-dffa3c331c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed9c5ada4eda4254a061acdfcc19f4e1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:25.371391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-70921733-2243-41d5-b714-dffa3c331c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed9c5ada4eda4254a061acdfcc19f4e1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:25.384349 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-70921733-2243-41d5-b714-dffa3c331c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed9c5ada4eda4254a061acdfcc19f4e1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:25.489027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 860/1713] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:03:24 2022] DELETE /identity/v3/users/ed9c5ada4eda4254a061acdfcc19f4e1 => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:26.245324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:26.408025 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16221b-3c1c-4d0c-9620-2587a4302aaf service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:26.430069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16221b-3c1c-4d0c-9620-2587a4302aaf service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:26.580006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb1dcc-2118-4eb1-a1bf-d97418141e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:26.582426 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb1dcc-2118-4eb1-a1bf-d97418141e72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:26.582426 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb1dcc-2118-4eb1-a1bf-d97418141e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:26.582811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eb1dcc-2118-4eb1-a1bf-d97418141e72 admin admin] PATH_INFO: `/v3/projects/0f9a34cf9c6e44f99b352c5a45dce759` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:26.583739 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb1dcc-2118-4eb1-a1bf-d97418141e72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f9a34cf9c6e44f99b352c5a45dce759)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:26.586939 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eb1dcc-2118-4eb1-a1bf-d97418141e72 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:26.814205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-18eb1dcc-2118-4eb1-a1bf-d97418141e72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f9a34cf9c6e44f99b352c5a45dce759', 'request_id': 'req-18eb1dcc-2118-4eb1-a1bf-d97418141e72'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:26.835348 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-18eb1dcc-2118-4eb1-a1bf-d97418141e72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f9a34cf9c6e44f99b352c5a45dce759', 'request_id': 'req-18eb1dcc-2118-4eb1-a1bf-d97418141e72'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:26.970182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-18eb1dcc-2118-4eb1-a1bf-d97418141e72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f9a34cf9c6e44f99b352c5a45dce759', 'request_id': 'req-18eb1dcc-2118-4eb1-a1bf-d97418141e72'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:26.988947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-18eb1dcc-2118-4eb1-a1bf-d97418141e72 admin admin] The token cache is being invalidate because project 0f9a34cf9c6e44f99b352c5a45dce759 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:03:26.989088 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-18eb1dcc-2118-4eb1-a1bf-d97418141e72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:26.993239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-18eb1dcc-2118-4eb1-a1bf-d97418141e72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:27.003916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 854/1714] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:03:26 2022] DELETE /identity/v3/projects/0f9a34cf9c6e44f99b352c5a45dce759 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:27.009494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70921733-2243-41d5-b714-dffa3c331c90 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:27.012591 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-70921733-2243-41d5-b714-dffa3c331c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:27.035685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70921733-2243-41d5-b714-dffa3c331c90 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:27.169100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae906bdf-e3f9-4baa-b790-4815d6733670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:27.169992 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae906bdf-e3f9-4baa-b790-4815d6733670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:27.170471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae906bdf-e3f9-4baa-b790-4815d6733670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:27.170886 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae906bdf-e3f9-4baa-b790-4815d6733670 admin admin] PATH_INFO: `/v3/users/41492c51c9fc4c248be80a8fa0b063e5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:27.173973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae906bdf-e3f9-4baa-b790-4815d6733670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41492c51c9fc4c248be80a8fa0b063e5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:27.176339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae906bdf-e3f9-4baa-b790-4815d6733670 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:27.321084 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:27.514438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae906bdf-e3f9-4baa-b790-4815d6733670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41492c51c9fc4c248be80a8fa0b063e5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:27.703713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae906bdf-e3f9-4baa-b790-4815d6733670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41492c51c9fc4c248be80a8fa0b063e5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:27.704453 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae906bdf-e3f9-4baa-b790-4815d6733670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41492c51c9fc4c248be80a8fa0b063e5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:27.705681 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae906bdf-e3f9-4baa-b790-4815d6733670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41492c51c9fc4c248be80a8fa0b063e5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:27.725193 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae906bdf-e3f9-4baa-b790-4815d6733670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41492c51c9fc4c248be80a8fa0b063e5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:27.759882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 861/1715] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:03:27 2022] DELETE /identity/v3/users/41492c51c9fc4c248be80a8fa0b063e5 => generated 0 bytes in 751 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:28.504592 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb1dcc-2118-4eb1-a1bf-d97418141e72 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:28.509068 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-18eb1dcc-2118-4eb1-a1bf-d97418141e72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:28.546093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eb1dcc-2118-4eb1-a1bf-d97418141e72 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:28.698194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6265e8-8f5b-40c6-8cbc-45e2c609e2a2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:28.698696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6265e8-8f5b-40c6-8cbc-45e2c609e2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:28.698696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6265e8-8f5b-40c6-8cbc-45e2c609e2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:28.730633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e260b4f-202c-4d3f-90bd-af9f932d7739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:28.731468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e260b4f-202c-4d3f-90bd-af9f932d7739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:28.734612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e260b4f-202c-4d3f-90bd-af9f932d7739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:28.735068 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e260b4f-202c-4d3f-90bd-af9f932d7739 admin admin] PATH_INFO: `/v3/projects/bcb1830b4d004072a2ae91a699f096bb` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:28.740897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e260b4f-202c-4d3f-90bd-af9f932d7739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcb1830b4d004072a2ae91a699f096bb)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:28.745285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e260b4f-202c-4d3f-90bd-af9f932d7739 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:28.748807 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ce6265e8-8f5b-40c6-8cbc-45e2c609e2a2 None None] MFA Rules not processed for user `14e51ac136874806b4b903235697b404`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:28.750039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ce6265e8-8f5b-40c6-8cbc-45e2c609e2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:28.924595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 862/1716] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:28.943173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78344a26-8545-41c6-b9a7-33d39bfec763 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:28.943457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78344a26-8545-41c6-b9a7-33d39bfec763 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:28.944007 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78344a26-8545-41c6-b9a7-33d39bfec763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:29.011412 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-78344a26-8545-41c6-b9a7-33d39bfec763 None None] MFA Rules not processed for user `14e51ac136874806b4b903235697b404`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:29.011412 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-78344a26-8545-41c6-b9a7-33d39bfec763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:29.031898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0e260b4f-202c-4d3f-90bd-af9f932d7739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcb1830b4d004072a2ae91a699f096bb', 'request_id': 'req-0e260b4f-202c-4d3f-90bd-af9f932d7739'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:29.048311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0e260b4f-202c-4d3f-90bd-af9f932d7739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcb1830b4d004072a2ae91a699f096bb', 'request_id': 'req-0e260b4f-202c-4d3f-90bd-af9f932d7739'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:29.187415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 863/1717] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:29.216193 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78344a26-8545-41c6-b9a7-33d39bfec763 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:29.220386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-78344a26-8545-41c6-b9a7-33d39bfec763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:29.257146 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0e260b4f-202c-4d3f-90bd-af9f932d7739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcb1830b4d004072a2ae91a699f096bb', 'request_id': 'req-0e260b4f-202c-4d3f-90bd-af9f932d7739'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:29.284168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0e260b4f-202c-4d3f-90bd-af9f932d7739 admin admin] The token cache is being invalidate because project bcb1830b4d004072a2ae91a699f096bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:03:29.284479 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0e260b4f-202c-4d3f-90bd-af9f932d7739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:29.287690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0e260b4f-202c-4d3f-90bd-af9f932d7739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:29.299225 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 855/1718] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:03:28 2022] DELETE /identity/v3/projects/bcb1830b4d004072a2ae91a699f096bb => generated 0 bytes in 795 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:29.306377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e29ae5-5c94-448c-8342-a8eca60b74e5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:29.306821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e29ae5-5c94-448c-8342-a8eca60b74e5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:29.307294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e29ae5-5c94-448c-8342-a8eca60b74e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:29.355352 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-07e29ae5-5c94-448c-8342-a8eca60b74e5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:29.356634 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-07e29ae5-5c94-448c-8342-a8eca60b74e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:29.376461 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:29.405143 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78344a26-8545-41c6-b9a7-33d39bfec763 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:29.432723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cbb350-2f66-4feb-ac07-5d8d9b638691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:29.433827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbb350-2f66-4feb-ac07-5d8d9b638691 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:29.434262 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbb350-2f66-4feb-ac07-5d8d9b638691 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:29.434929 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbb350-2f66-4feb-ac07-5d8d9b638691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:29.437504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a4cbb350-2f66-4feb-ac07-5d8d9b638691 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:29.458312 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbb350-2f66-4feb-ac07-5d8d9b638691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:29.461144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cbb350-2f66-4feb-ac07-5d8d9b638691 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:29.490230 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 864/1719] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:29 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:29.493156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f9943-262a-4bc1-8bab-96b901d31e19 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:29.493446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f9943-262a-4bc1-8bab-96b901d31e19 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:29.493923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646f9943-262a-4bc1-8bab-96b901d31e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:29.579516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 856/1720] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:29.589357 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60ac2f-a4f5-4ec2-bec0-b9ffad8466ce None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:29.589761 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60ac2f-a4f5-4ec2-bec0-b9ffad8466ce None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:29.590139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc60ac2f-a4f5-4ec2-bec0-b9ffad8466ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:29.611969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-646f9943-262a-4bc1-8bab-96b901d31e19 None None] MFA Rules not processed for user `97ed136f68c540abbdaf04cbe34a682e`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:29.613290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-646f9943-262a-4bc1-8bab-96b901d31e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:29.644623 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fc60ac2f-a4f5-4ec2-bec0-b9ffad8466ce None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:29.646079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fc60ac2f-a4f5-4ec2-bec0-b9ffad8466ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:29.681033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 857/1721] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:29.683189 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60ac2f-a4f5-4ec2-bec0-b9ffad8466ce None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:29.707463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc60ac2f-a4f5-4ec2-bec0-b9ffad8466ce None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:29.716643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bc384-c874-4d4e-a753-162332cbecb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:29.718008 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bc384-c874-4d4e-a753-162332cbecb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:29.718465 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bc384-c874-4d4e-a753-162332cbecb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:29.718829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bc384-c874-4d4e-a753-162332cbecb6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:29.719842 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bc384-c874-4d4e-a753-162332cbecb6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:29.720384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bc384-c874-4d4e-a753-162332cbecb6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:29.722829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bc384-c874-4d4e-a753-162332cbecb6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:29.757100 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 858/1722] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:03:29.758243 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bc384-c874-4d4e-a753-162332cbecb6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:29.770753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bc384-c874-4d4e-a753-162332cbecb6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:29.782589 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb067f-33ff-421e-b744-2cd08ca415a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:29.785790 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb067f-33ff-421e-b744-2cd08ca415a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:29.787504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb067f-33ff-421e-b744-2cd08ca415a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:29.789485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb067f-33ff-421e-b744-2cd08ca415a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:29.792810 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb067f-33ff-421e-b744-2cd08ca415a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:29.794792 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb067f-33ff-421e-b744-2cd08ca415a4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:29.902277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 865/1723] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:29.926484 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eefb5-804c-4f5a-8e0f-bf724eb110e1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:29.927001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eefb5-804c-4f5a-8e0f-bf724eb110e1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:29.927423 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57eefb5-804c-4f5a-8e0f-bf724eb110e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:30.009308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a57eefb5-804c-4f5a-8e0f-bf724eb110e1 None None] MFA Rules not processed for user `97ed136f68c540abbdaf04cbe34a682e`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:30.011843 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a57eefb5-804c-4f5a-8e0f-bf724eb110e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:30.046968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 866/1724] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:30.055921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eefb5-804c-4f5a-8e0f-bf724eb110e1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:30.059518 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a57eefb5-804c-4f5a-8e0f-bf724eb110e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:30.079184 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 859/1725] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:29 2022] POST /identity/v3/projects => generated 385 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:30.083237 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb067f-33ff-421e-b744-2cd08ca415a4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:30.125776 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57eefb5-804c-4f5a-8e0f-bf724eb110e1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:30.275099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb067f-33ff-421e-b744-2cd08ca415a4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:30.303319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dff518-5f27-4a3f-8e5c-284005fa544f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:30.303958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dff518-5f27-4a3f-8e5c-284005fa544f admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:30.304193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dff518-5f27-4a3f-8e5c-284005fa544f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:30.304474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dff518-5f27-4a3f-8e5c-284005fa544f admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:30.305126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dff518-5f27-4a3f-8e5c-284005fa544f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:30.310615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dff518-5f27-4a3f-8e5c-284005fa544f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:30.339064 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbcdb1f-1a9f-4250-9965-0c458208008d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:30.339725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcdb1f-1a9f-4250-9965-0c458208008d service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:30.340012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcdb1f-1a9f-4250-9965-0c458208008d service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:30.340289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbcdb1f-1a9f-4250-9965-0c458208008d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:30.512684 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 860/1726] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:30 2022] POST /identity/v3/users => generated 293 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:30.515612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dff518-5f27-4a3f-8e5c-284005fa544f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:30.531896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcdb1f-1a9f-4250-9965-0c458208008d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:30.531896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbcdb1f-1a9f-4250-9965-0c458208008d service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:30.575526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 867/1727] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:30 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:30.580737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dff518-5f27-4a3f-8e5c-284005fa544f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:30.588432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:30.594292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98351a0a-514c-4246-936f-aa5bf6a9a790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:30.595129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98351a0a-514c-4246-936f-aa5bf6a9a790 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:30.595567 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98351a0a-514c-4246-936f-aa5bf6a9a790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:30.596020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98351a0a-514c-4246-936f-aa5bf6a9a790 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:30.596823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98351a0a-514c-4246-936f-aa5bf6a9a790 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:30.598844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98351a0a-514c-4246-936f-aa5bf6a9a790 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:30.622352 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e0d89-5479-4493-a8a3-1636a5495691 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:30.622643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e0d89-5479-4493-a8a3-1636a5495691 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:30.622917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07e0d89-5479-4493-a8a3-1636a5495691 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:30.626848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 861/1728] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:30.628258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 868/1729] 10.222.0.67 () {54 vars in 912 bytes} [Mon Sep 26 18:03:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:03:30.633875 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98351a0a-514c-4246-936f-aa5bf6a9a790 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:30.643004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb9088-20c5-49f8-b592-92e76ff43d89 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:30.643004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb9088-20c5-49f8-b592-92e76ff43d89 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:30.643004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fb9088-20c5-49f8-b592-92e76ff43d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:30.670983 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98351a0a-514c-4246-936f-aa5bf6a9a790 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:30.693647 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b0d0f-df16-412b-841c-adf140b96c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:30.694281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b0d0f-df16-412b-841c-adf140b96c57 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:30.694759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b0d0f-df16-412b-841c-adf140b96c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:30.695270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b0d0f-df16-412b-841c-adf140b96c57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:30.702560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b0d0f-df16-412b-841c-adf140b96c57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:30.704252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b0d0f-df16-412b-841c-adf140b96c57 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:30.712377 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-18fb9088-20c5-49f8-b592-92e76ff43d89 None None] MFA Rules not processed for user `ef7e0b772a284e7da4fae7d73f34d3de`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:30.717101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-18fb9088-20c5-49f8-b592-92e76ff43d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:30.733860 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 862/1730] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:30.734907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b0d0f-df16-412b-841c-adf140b96c57 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:30.760135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b0d0f-df16-412b-841c-adf140b96c57 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:30.770835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cae475-a7be-46aa-8fc5-a17191b1d906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:30.771544 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cae475-a7be-46aa-8fc5-a17191b1d906 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:30.773680 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cae475-a7be-46aa-8fc5-a17191b1d906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:30.773981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cae475-a7be-46aa-8fc5-a17191b1d906 admin admin] PATH_INFO: `/v3/projects/1cc67f01334e4295ac91813ba4598261/users/6a8b0537bc1e418992b3313389e2bc61/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:30.829698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 869/1731] 10.222.0.67 () {58 vars in 1009 bytes} [Mon Sep 26 18:03:30 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:30.839927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cae475-a7be-46aa-8fc5-a17191b1d906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cc67f01334e4295ac91813ba4598261, user_id=6a8b0537bc1e418992b3313389e2bc61, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:30.844033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cae475-a7be-46aa-8fc5-a17191b1d906 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:30.928243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab996b-ce18-48e9-8b3c-73f66a27f75f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:30.928758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab996b-ce18-48e9-8b3c-73f66a27f75f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:30.929081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab996b-ce18-48e9-8b3c-73f66a27f75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:30.974028 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7eab996b-ce18-48e9-8b3c-73f66a27f75f None None] MFA Rules not processed for user `0bede3a601f941e7abc9fd16983679b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:30.990428 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7eab996b-ce18-48e9-8b3c-73f66a27f75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:31.072193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 863/1732] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:30 2022] PUT /identity/v3/projects/1cc67f01334e4295ac91813ba4598261/users/6a8b0537bc1e418992b3313389e2bc61/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:31.073511 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cae475-a7be-46aa-8fc5-a17191b1d906 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:31.078209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-64cae475-a7be-46aa-8fc5-a17191b1d906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:31.104705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 870/1733] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:31.107071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e379ef-baf7-4d79-ab28-b169bb9980cb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:31.108173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e379ef-baf7-4d79-ab28-b169bb9980cb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:31.108913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e379ef-baf7-4d79-ab28-b169bb9980cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:31.191914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-57e379ef-baf7-4d79-ab28-b169bb9980cb None None] MFA Rules not processed for user `6a8b0537bc1e418992b3313389e2bc61`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:31.196473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-57e379ef-baf7-4d79-ab28-b169bb9980cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:31.246319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cae475-a7be-46aa-8fc5-a17191b1d906 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:31.252354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f27c36-016a-4579-b9bf-109743580621 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:31.252985 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f27c36-016a-4579-b9bf-109743580621 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:31.253266 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f27c36-016a-4579-b9bf-109743580621 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:31.253595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f27c36-016a-4579-b9bf-109743580621 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:31.268762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f27c36-016a-4579-b9bf-109743580621 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:31.270222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f27c36-016a-4579-b9bf-109743580621 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:31.284351 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 864/1734] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:31.287778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c2a36-de99-4527-8887-26c7d0831e72 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:31.288097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c2a36-de99-4527-8887-26c7d0831e72 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:31.288471 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c2a36-de99-4527-8887-26c7d0831e72 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:31.298261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 865/1735] 10.222.0.67 () {54 vars in 912 bytes} [Mon Sep 26 18:03:31 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:03:31.299668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09134e1-47c7-4d76-a163-2f128848364e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:31.300035 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09134e1-47c7-4d76-a163-2f128848364e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:31.300265 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09134e1-47c7-4d76-a163-2f128848364e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:31.312977 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 871/1736] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:31.317918 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e379ef-baf7-4d79-ab28-b169bb9980cb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:31.332845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b09134e1-47c7-4d76-a163-2f128848364e None None] MFA Rules not processed for user `ef7e0b772a284e7da4fae7d73f34d3de`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:31.334176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b09134e1-47c7-4d76-a163-2f128848364e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:31.374356 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 866/1737] 10.222.0.67 () {58 vars in 1009 bytes} [Mon Sep 26 18:03:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:31.381412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09134e1-47c7-4d76-a163-2f128848364e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:31.386872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b09134e1-47c7-4d76-a163-2f128848364e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:31.403533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e379ef-baf7-4d79-ab28-b169bb9980cb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:31.409177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa037a2-c454-4d3e-b5af-5d9d6c0ebb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:31.409731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa037a2-c454-4d3e-b5af-5d9d6c0ebb9a admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:31.410185 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa037a2-c454-4d3e-b5af-5d9d6c0ebb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:31.410506 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa037a2-c454-4d3e-b5af-5d9d6c0ebb9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:31.411209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa037a2-c454-4d3e-b5af-5d9d6c0ebb9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:31.413398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa037a2-c454-4d3e-b5af-5d9d6c0ebb9a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:31.420208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09134e1-47c7-4d76-a163-2f128848364e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:31.426992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889235a-69a6-4cec-b34f-40706d66b2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:31.427483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889235a-69a6-4cec-b34f-40706d66b2e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:31.427806 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889235a-69a6-4cec-b34f-40706d66b2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:31.428086 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889235a-69a6-4cec-b34f-40706d66b2e2 admin admin] PATH_INFO: `/v3/users/8a6b49e8387e458ba5e5dd40575479df` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:31.429818 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889235a-69a6-4cec-b34f-40706d66b2e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a6b49e8387e458ba5e5dd40575479df)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:31.434137 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889235a-69a6-4cec-b34f-40706d66b2e2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:31.657019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 872/1738] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:31 2022] POST /identity/v3/projects => generated 387 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:31.657019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa037a2-c454-4d3e-b5af-5d9d6c0ebb9a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:31.744388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa037a2-c454-4d3e-b5af-5d9d6c0ebb9a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:31.750055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6329df4-f68a-42ce-835e-c69c2bb42ec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:31.750699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6329df4-f68a-42ce-835e-c69c2bb42ec8 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:31.751169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6329df4-f68a-42ce-835e-c69c2bb42ec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:31.751829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6329df4-f68a-42ce-835e-c69c2bb42ec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:31.760706 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6329df4-f68a-42ce-835e-c69c2bb42ec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:31.767219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6329df4-f68a-42ce-835e-c69c2bb42ec8 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:31.771138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 873/1739] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:31.772207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6329df4-f68a-42ce-835e-c69c2bb42ec8 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:31.874037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6329df4-f68a-42ce-835e-c69c2bb42ec8 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:31.883461 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21486584-d920-485b-bf8b-3bf05b947002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:31.884437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21486584-d920-485b-bf8b-3bf05b947002 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:31.884722 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21486584-d920-485b-bf8b-3bf05b947002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:31.885000 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21486584-d920-485b-bf8b-3bf05b947002 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:31.885979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21486584-d920-485b-bf8b-3bf05b947002 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:31.888589 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21486584-d920-485b-bf8b-3bf05b947002 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:32.043676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9889235a-69a6-4cec-b34f-40706d66b2e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a6b49e8387e458ba5e5dd40575479df'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:32.086210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 874/1740] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:31 2022] POST /identity/v3/users => generated 300 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:32.088386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21486584-d920-485b-bf8b-3bf05b947002 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:32.107986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21486584-d920-485b-bf8b-3bf05b947002 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:32.114241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f5119-25ae-4b94-bd53-3cee402a086b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:32.115157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f5119-25ae-4b94-bd53-3cee402a086b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:32.115607 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f5119-25ae-4b94-bd53-3cee402a086b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:32.116983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5f5119-25ae-4b94-bd53-3cee402a086b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:32.118177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f5119-25ae-4b94-bd53-3cee402a086b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:32.122567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5f5119-25ae-4b94-bd53-3cee402a086b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:32.140475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 875/1741] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:32.141460 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f5119-25ae-4b94-bd53-3cee402a086b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:32.143202 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9a5f5119-25ae-4b94-bd53-3cee402a086b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:32.181240 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9889235a-69a6-4cec-b34f-40706d66b2e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a6b49e8387e458ba5e5dd40575479df'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:32.184062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9889235a-69a6-4cec-b34f-40706d66b2e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a6b49e8387e458ba5e5dd40575479df'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:32.188170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9889235a-69a6-4cec-b34f-40706d66b2e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a6b49e8387e458ba5e5dd40575479df'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:32.203413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:32.221879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9889235a-69a6-4cec-b34f-40706d66b2e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a6b49e8387e458ba5e5dd40575479df'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:32.251268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 867/1742] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:03:31 2022] DELETE /identity/v3/users/8a6b49e8387e458ba5e5dd40575479df => generated 0 bytes in 870 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:32.252424 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889235a-69a6-4cec-b34f-40706d66b2e2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:32.259592 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9889235a-69a6-4cec-b34f-40706d66b2e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:32.317012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5f5119-25ae-4b94-bd53-3cee402a086b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:32.402180 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889235a-69a6-4cec-b34f-40706d66b2e2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:32.409117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e23330-a1d3-4f9a-811a-b6cf5f57434c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:32.409873 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e23330-a1d3-4f9a-811a-b6cf5f57434c admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:32.410452 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e23330-a1d3-4f9a-811a-b6cf5f57434c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:32.410739 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e23330-a1d3-4f9a-811a-b6cf5f57434c admin admin] PATH_INFO: `/v3/projects/ffdbc5fb68b0454d9dee49665b736efd/users/ff24adc6a0bb48e9b0220c584c126194/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:32.433172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5728a12-bed9-4859-94fd-af9130378b52 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:32.434019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5728a12-bed9-4859-94fd-af9130378b52 service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:32.434344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e23330-a1d3-4f9a-811a-b6cf5f57434c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffdbc5fb68b0454d9dee49665b736efd, user_id=ff24adc6a0bb48e9b0220c584c126194, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:32.434512 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5728a12-bed9-4859-94fd-af9130378b52 service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:32.434628 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5728a12-bed9-4859-94fd-af9130378b52 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:32.437851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e5728a12-bed9-4859-94fd-af9130378b52 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:32.438811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e23330-a1d3-4f9a-811a-b6cf5f57434c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:32.456660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5728a12-bed9-4859-94fd-af9130378b52 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:32.458280 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5728a12-bed9-4859-94fd-af9130378b52 service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:32.479743 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 876/1743] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:32.481118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e6e1b-0b9f-45dc-b3fd-9b82c6d2d4f4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:32.483525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e6e1b-0b9f-45dc-b3fd-9b82c6d2d4f4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:32.483849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e6e1b-0b9f-45dc-b3fd-9b82c6d2d4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:32.524831 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-085e6e1b-0b9f-45dc-b3fd-9b82c6d2d4f4 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:32.529138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-085e6e1b-0b9f-45dc-b3fd-9b82c6d2d4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:32.543732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 877/1744] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:32.556892 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e6e1b-0b9f-45dc-b3fd-9b82c6d2d4f4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:32.558090 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-085e6e1b-0b9f-45dc-b3fd-9b82c6d2d4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:32.640952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e6e1b-0b9f-45dc-b3fd-9b82c6d2d4f4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:32.650940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ae052-2f43-4b12-bf94-2ebb4c7f2771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:32.652419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ae052-2f43-4b12-bf94-2ebb4c7f2771 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:32.652419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ae052-2f43-4b12-bf94-2ebb4c7f2771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:32.652419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361ae052-2f43-4b12-bf94-2ebb4c7f2771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:32.665027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ae052-2f43-4b12-bf94-2ebb4c7f2771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:32.669077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 868/1745] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:32 2022] PUT /identity/v3/projects/ffdbc5fb68b0454d9dee49665b736efd/users/ff24adc6a0bb48e9b0220c584c126194/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:32.669292 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361ae052-2f43-4b12-bf94-2ebb4c7f2771 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:32.670590 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e23330-a1d3-4f9a-811a-b6cf5f57434c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:32.686767 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 878/1746] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:32.766932 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e23330-a1d3-4f9a-811a-b6cf5f57434c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:32.773493 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc78ab3-0bd2-4f9d-8498-319f957d90e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:32.774132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc78ab3-0bd2-4f9d-8498-319f957d90e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:32.774563 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc78ab3-0bd2-4f9d-8498-319f957d90e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:32.774986 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc78ab3-0bd2-4f9d-8498-319f957d90e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:32.776148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc78ab3-0bd2-4f9d-8498-319f957d90e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:32.778261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc78ab3-0bd2-4f9d-8498-319f957d90e0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:32.793323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 869/1747] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:03:32.796583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ae052-2f43-4b12-bf94-2ebb4c7f2771 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:32.863164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361ae052-2f43-4b12-bf94-2ebb4c7f2771 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:32.894857 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11718f1d-caf5-4552-a692-ab44b8af107a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:32.895834 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11718f1d-caf5-4552-a692-ab44b8af107a admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:32.896247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11718f1d-caf5-4552-a692-ab44b8af107a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:32.900897 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11718f1d-caf5-4552-a692-ab44b8af107a admin admin] PATH_INFO: `/v3/projects/ffdbc5fb68b0454d9dee49665b736efd/users/ff24adc6a0bb48e9b0220c584c126194/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:32.919368 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11718f1d-caf5-4552-a692-ab44b8af107a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffdbc5fb68b0454d9dee49665b736efd, user_id=ff24adc6a0bb48e9b0220c584c126194, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:32.933921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11718f1d-caf5-4552-a692-ab44b8af107a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:32.971809 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 879/1748] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:32 2022] PUT /identity/v3/projects/ffdbc5fb68b0454d9dee49665b736efd/users/ff24adc6a0bb48e9b0220c584c126194/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:32.975004 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4d075-7b6e-4e26-b65d-d33162798ccb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:32.980333 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4d075-7b6e-4e26-b65d-d33162798ccb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:32.981092 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4d075-7b6e-4e26-b65d-d33162798ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:33.043088 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11718f1d-caf5-4552-a692-ab44b8af107a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:33.045756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-11718f1d-caf5-4552-a692-ab44b8af107a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:33.151730 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-21f4d075-7b6e-4e26-b65d-d33162798ccb None None] MFA Rules not processed for user `ff24adc6a0bb48e9b0220c584c126194`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:33.154336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-21f4d075-7b6e-4e26-b65d-d33162798ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:33.208312 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11718f1d-caf5-4552-a692-ab44b8af107a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:33.223081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067a505-6ddc-401d-939e-c12e2a666449 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:33.224107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067a505-6ddc-401d-939e-c12e2a666449 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:33.224514 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067a505-6ddc-401d-939e-c12e2a666449 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:33.224898 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067a505-6ddc-401d-939e-c12e2a666449 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:33.234614 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1067a505-6ddc-401d-939e-c12e2a666449 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:33.366473 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 870/1749] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:32 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 393 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:33.370127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffa1ec-9798-49bb-a7c8-0415423ae3cb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:33.370127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffa1ec-9798-49bb-a7c8-0415423ae3cb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:33.370127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffa1ec-9798-49bb-a7c8-0415423ae3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:33.411268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-15ffa1ec-9798-49bb-a7c8-0415423ae3cb None None] MFA Rules not processed for user `97ed136f68c540abbdaf04cbe34a682e`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:33.421252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-15ffa1ec-9798-49bb-a7c8-0415423ae3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:33.501097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:33.539727 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067a505-6ddc-401d-939e-c12e2a666449 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:33.548973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067a505-6ddc-401d-939e-c12e2a666449 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:33.605829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 880/1750] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:33 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:03:33.610132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067a505-6ddc-401d-939e-c12e2a666449 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:33.617623 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1067a505-6ddc-401d-939e-c12e2a666449 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:33.641873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067a505-6ddc-401d-939e-c12e2a666449 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:33.645742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 871/1751] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:33 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:33.652665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396f78d-7416-46b6-a101-ce40795163b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:33.653404 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396f78d-7416-46b6-a101-ce40795163b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:33.654395 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396f78d-7416-46b6-a101-ce40795163b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:33.654827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9396f78d-7416-46b6-a101-ce40795163b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:33.667122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396f78d-7416-46b6-a101-ce40795163b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:33.669188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9396f78d-7416-46b6-a101-ce40795163b7 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:33.692718 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 881/1752] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:33.808870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffa1ec-9798-49bb-a7c8-0415423ae3cb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:33.893746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396f78d-7416-46b6-a101-ce40795163b7 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:33.895992 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9396f78d-7416-46b6-a101-ce40795163b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:33.909804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:34.014625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffa1ec-9798-49bb-a7c8-0415423ae3cb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:34.052270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8b6ae8-4f7a-40b5-92cd-8f59527e342c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:34.052270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b6ae8-4f7a-40b5-92cd-8f59527e342c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:34.052270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b6ae8-4f7a-40b5-92cd-8f59527e342c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:34.052270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8b6ae8-4f7a-40b5-92cd-8f59527e342c admin admin] PATH_INFO: `/v3/users/6a8b0537bc1e418992b3313389e2bc61` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:34.054233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b6ae8-4f7a-40b5-92cd-8f59527e342c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a8b0537bc1e418992b3313389e2bc61)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:34.077374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8b6ae8-4f7a-40b5-92cd-8f59527e342c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:34.224084 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9396f78d-7416-46b6-a101-ce40795163b7 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:34.230894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192d491-2fc1-4a06-9a10-dbea234c89c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:34.231815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192d491-2fc1-4a06-9a10-dbea234c89c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:34.232173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192d491-2fc1-4a06-9a10-dbea234c89c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:34.232508 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192d491-2fc1-4a06-9a10-dbea234c89c7 admin admin] PATH_INFO: `/v3/projects/69f4982ba2e5443d9587c84439068bb0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:34.234598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192d491-2fc1-4a06-9a10-dbea234c89c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69f4982ba2e5443d9587c84439068bb0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:34.236914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192d491-2fc1-4a06-9a10-dbea234c89c7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:34.438467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ce8b6ae8-4f7a-40b5-92cd-8f59527e342c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a8b0537bc1e418992b3313389e2bc61'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:34.459193 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5192d491-2fc1-4a06-9a10-dbea234c89c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69f4982ba2e5443d9587c84439068bb0', 'request_id': 'req-5192d491-2fc1-4a06-9a10-dbea234c89c7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:34.483809 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5192d491-2fc1-4a06-9a10-dbea234c89c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69f4982ba2e5443d9587c84439068bb0', 'request_id': 'req-5192d491-2fc1-4a06-9a10-dbea234c89c7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:34.824290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5192d491-2fc1-4a06-9a10-dbea234c89c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69f4982ba2e5443d9587c84439068bb0', 'request_id': 'req-5192d491-2fc1-4a06-9a10-dbea234c89c7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:34.835073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ce8b6ae8-4f7a-40b5-92cd-8f59527e342c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a8b0537bc1e418992b3313389e2bc61'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:34.838207 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ce8b6ae8-4f7a-40b5-92cd-8f59527e342c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a8b0537bc1e418992b3313389e2bc61'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:34.839933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ce8b6ae8-4f7a-40b5-92cd-8f59527e342c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a8b0537bc1e418992b3313389e2bc61'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:34.859930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5192d491-2fc1-4a06-9a10-dbea234c89c7 admin admin] The token cache is being invalidate because project 69f4982ba2e5443d9587c84439068bb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:03:34.862756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5192d491-2fc1-4a06-9a10-dbea234c89c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:34.876608 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5192d491-2fc1-4a06-9a10-dbea234c89c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:34.879975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 882/1753] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:03:33 2022] DELETE /identity/v3/projects/69f4982ba2e5443d9587c84439068bb0 => generated 0 bytes in 986 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:34.884102 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192d491-2fc1-4a06-9a10-dbea234c89c7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:34.886189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5192d491-2fc1-4a06-9a10-dbea234c89c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:34.891957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ce8b6ae8-4f7a-40b5-92cd-8f59527e342c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a8b0537bc1e418992b3313389e2bc61'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:34.933224 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 872/1754] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:03:33 2022] DELETE /identity/v3/users/6a8b0537bc1e418992b3313389e2bc61 => generated 0 bytes in 1125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:34.938106 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75de72a-3178-4163-842a-d31f16c695b6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:34.938492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75de72a-3178-4163-842a-d31f16c695b6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:34.938844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75de72a-3178-4163-842a-d31f16c695b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:34.984557 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b75de72a-3178-4163-842a-d31f16c695b6 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:34.987558 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b75de72a-3178-4163-842a-d31f16c695b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:35.068700 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192d491-2fc1-4a06-9a10-dbea234c89c7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:35.167507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 873/1755] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:35.179704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75de72a-3178-4163-842a-d31f16c695b6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:35.183213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b75de72a-3178-4163-842a-d31f16c695b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:35.229459 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f5ed5-d966-458c-b6b1-b4715ea66ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:35.243950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f5ed5-d966-458c-b6b1-b4715ea66ec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:35.243950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f5ed5-d966-458c-b6b1-b4715ea66ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:35.243950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357f5ed5-d966-458c-b6b1-b4715ea66ec5 admin admin] PATH_INFO: `/v3/users/333444f04b5c4a5bbafe075ccc0ebcc9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:35.243950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f5ed5-d966-458c-b6b1-b4715ea66ec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=333444f04b5c4a5bbafe075ccc0ebcc9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:35.243950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357f5ed5-d966-458c-b6b1-b4715ea66ec5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:35.398055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75de72a-3178-4163-842a-d31f16c695b6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:35.402914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca124670-de60-4c9f-9481-eddbc04684c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:35.403780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca124670-de60-4c9f-9481-eddbc04684c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:35.404539 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca124670-de60-4c9f-9481-eddbc04684c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:35.404962 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca124670-de60-4c9f-9481-eddbc04684c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:35.418368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca124670-de60-4c9f-9481-eddbc04684c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:35.420289 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca124670-de60-4c9f-9481-eddbc04684c7 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:35.433189 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 874/1756] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:35.653456 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-357f5ed5-d966-458c-b6b1-b4715ea66ec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333444f04b5c4a5bbafe075ccc0ebcc9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:35.758411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-357f5ed5-d966-458c-b6b1-b4715ea66ec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333444f04b5c4a5bbafe075ccc0ebcc9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:35.768276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-357f5ed5-d966-458c-b6b1-b4715ea66ec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333444f04b5c4a5bbafe075ccc0ebcc9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:35.769323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-357f5ed5-d966-458c-b6b1-b4715ea66ec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333444f04b5c4a5bbafe075ccc0ebcc9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:35.780242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-357f5ed5-d966-458c-b6b1-b4715ea66ec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333444f04b5c4a5bbafe075ccc0ebcc9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:35.810602 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 883/1757] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:03:34 2022] DELETE /identity/v3/users/333444f04b5c4a5bbafe075ccc0ebcc9 => generated 0 bytes in 927 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:36.219904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca124670-de60-4c9f-9481-eddbc04684c7 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:36.219904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ca124670-de60-4c9f-9481-eddbc04684c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:36.229922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca124670-de60-4c9f-9481-eddbc04684c7 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:36.299185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14366b24-7372-482b-a06d-67635ef5ef90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:36.299960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14366b24-7372-482b-a06d-67635ef5ef90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:36.300241 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14366b24-7372-482b-a06d-67635ef5ef90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:36.300522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14366b24-7372-482b-a06d-67635ef5ef90 admin admin] PATH_INFO: `/v3/projects/1cc67f01334e4295ac91813ba4598261` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:36.301668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14366b24-7372-482b-a06d-67635ef5ef90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cc67f01334e4295ac91813ba4598261)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:36.303626 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14366b24-7372-482b-a06d-67635ef5ef90 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:36.486689 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-14366b24-7372-482b-a06d-67635ef5ef90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cc67f01334e4295ac91813ba4598261', 'request_id': 'req-14366b24-7372-482b-a06d-67635ef5ef90'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:36.493644 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-14366b24-7372-482b-a06d-67635ef5ef90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cc67f01334e4295ac91813ba4598261', 'request_id': 'req-14366b24-7372-482b-a06d-67635ef5ef90'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:36.524170 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f5ed5-d966-458c-b6b1-b4715ea66ec5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:36.526127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-357f5ed5-d966-458c-b6b1-b4715ea66ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:36.546377 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357f5ed5-d966-458c-b6b1-b4715ea66ec5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:36.598184 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-14366b24-7372-482b-a06d-67635ef5ef90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cc67f01334e4295ac91813ba4598261', 'request_id': 'req-14366b24-7372-482b-a06d-67635ef5ef90'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:36.612107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-14366b24-7372-482b-a06d-67635ef5ef90 admin admin] The token cache is being invalidate because project 1cc67f01334e4295ac91813ba4598261 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:03:36.612291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a22f53a-3e93-4038-9c58-fd05fc11fb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:36.612417 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-14366b24-7372-482b-a06d-67635ef5ef90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:36.612786 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22f53a-3e93-4038-9c58-fd05fc11fb72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:36.613073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22f53a-3e93-4038-9c58-fd05fc11fb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:36.613345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a22f53a-3e93-4038-9c58-fd05fc11fb72 admin admin] PATH_INFO: `/v3/projects/357d06436ed8423a8d49de78caecf243` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:36.613449 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-14366b24-7372-482b-a06d-67635ef5ef90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:36.614463 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a22f53a-3e93-4038-9c58-fd05fc11fb72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=357d06436ed8423a8d49de78caecf243)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:36.617959 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a22f53a-3e93-4038-9c58-fd05fc11fb72 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:36.619053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 875/1758] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:03:36 2022] DELETE /identity/v3/projects/1cc67f01334e4295ac91813ba4598261 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:36.620054 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14366b24-7372-482b-a06d-67635ef5ef90 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:36.621508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-14366b24-7372-482b-a06d-67635ef5ef90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:36.636041 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14366b24-7372-482b-a06d-67635ef5ef90 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:36.640854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace15b4a-e88c-4fe9-9a28-8b88e92b8dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:36.641564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace15b4a-e88c-4fe9-9a28-8b88e92b8dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:36.642439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace15b4a-e88c-4fe9-9a28-8b88e92b8dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:36.642942 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace15b4a-e88c-4fe9-9a28-8b88e92b8dc1 admin admin] PATH_INFO: `/v3/users/ff24adc6a0bb48e9b0220c584c126194` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:36.644719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace15b4a-e88c-4fe9-9a28-8b88e92b8dc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff24adc6a0bb48e9b0220c584c126194)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:36.646427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace15b4a-e88c-4fe9-9a28-8b88e92b8dc1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:36.826814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:36.931492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ace15b4a-e88c-4fe9-9a28-8b88e92b8dc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff24adc6a0bb48e9b0220c584c126194'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:36.933601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a22f53a-3e93-4038-9c58-fd05fc11fb72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '357d06436ed8423a8d49de78caecf243', 'request_id': 'req-9a22f53a-3e93-4038-9c58-fd05fc11fb72'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:36.941131 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a22f53a-3e93-4038-9c58-fd05fc11fb72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '357d06436ed8423a8d49de78caecf243', 'request_id': 'req-9a22f53a-3e93-4038-9c58-fd05fc11fb72'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:37.052358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ace15b4a-e88c-4fe9-9a28-8b88e92b8dc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff24adc6a0bb48e9b0220c584c126194'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:37.055191 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ace15b4a-e88c-4fe9-9a28-8b88e92b8dc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff24adc6a0bb48e9b0220c584c126194'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:37.056837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ace15b4a-e88c-4fe9-9a28-8b88e92b8dc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff24adc6a0bb48e9b0220c584c126194'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:37.075320 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ace15b4a-e88c-4fe9-9a28-8b88e92b8dc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff24adc6a0bb48e9b0220c584c126194'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:37.094242 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 876/1759] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:03:36 2022] DELETE /identity/v3/users/ff24adc6a0bb48e9b0220c584c126194 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:37.108833 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a22f53a-3e93-4038-9c58-fd05fc11fb72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '357d06436ed8423a8d49de78caecf243', 'request_id': 'req-9a22f53a-3e93-4038-9c58-fd05fc11fb72'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:37.129814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a22f53a-3e93-4038-9c58-fd05fc11fb72 admin admin] The token cache is being invalidate because project 357d06436ed8423a8d49de78caecf243 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:03:37.130256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a22f53a-3e93-4038-9c58-fd05fc11fb72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:37.130998 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9a22f53a-3e93-4038-9c58-fd05fc11fb72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:37.133489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 884/1760] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:03:36 2022] DELETE /identity/v3/projects/357d06436ed8423a8d49de78caecf243 => generated 0 bytes in 610 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:37.138404 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d42d3a-053b-4c65-9599-beb2c5a1b8e7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:37.138682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d42d3a-053b-4c65-9599-beb2c5a1b8e7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:37.138950 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d42d3a-053b-4c65-9599-beb2c5a1b8e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:37.159813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c2d42d3a-053b-4c65-9599-beb2c5a1b8e7 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:37.163354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c2d42d3a-053b-4c65-9599-beb2c5a1b8e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:37.265746 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 877/1761] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:37.274487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6d0ac-ae39-4fc4-9ae7-bbdc9c67a51c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:37.274834 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6d0ac-ae39-4fc4-9ae7-bbdc9c67a51c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:37.275277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc6d0ac-ae39-4fc4-9ae7-bbdc9c67a51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:37.300579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4bc6d0ac-ae39-4fc4-9ae7-bbdc9c67a51c None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:37.302002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4bc6d0ac-ae39-4fc4-9ae7-bbdc9c67a51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:37.321811 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 885/1762] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:37.323185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d42d3a-053b-4c65-9599-beb2c5a1b8e7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:37.333531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d42d3a-053b-4c65-9599-beb2c5a1b8e7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:37.340085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12a4c2-27e0-49e2-b3c1-1910e4b65bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:37.341155 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12a4c2-27e0-49e2-b3c1-1910e4b65bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:37.341441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12a4c2-27e0-49e2-b3c1-1910e4b65bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:37.341759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee12a4c2-27e0-49e2-b3c1-1910e4b65bc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:37.342357 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12a4c2-27e0-49e2-b3c1-1910e4b65bc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:37.342765 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12a4c2-27e0-49e2-b3c1-1910e4b65bc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:37.344316 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee12a4c2-27e0-49e2-b3c1-1910e4b65bc3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:37.357935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 878/1763] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:03:37.373933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6d0ac-ae39-4fc4-9ae7-bbdc9c67a51c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:37.384446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc6d0ac-ae39-4fc4-9ae7-bbdc9c67a51c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:37.390340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c4471-4b73-49fe-97a8-116da4a81cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:37.390449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c4471-4b73-49fe-97a8-116da4a81cf0 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:37.390817 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c4471-4b73-49fe-97a8-116da4a81cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:37.391166 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1c4471-4b73-49fe-97a8-116da4a81cf0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:37.391998 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c4471-4b73-49fe-97a8-116da4a81cf0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:37.394011 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1c4471-4b73-49fe-97a8-116da4a81cf0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:37.552245 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 886/1764] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:37 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:37.556380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12a4c2-27e0-49e2-b3c1-1910e4b65bc3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:37.567267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee12a4c2-27e0-49e2-b3c1-1910e4b65bc3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:37.633691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ceb55-51a6-41bf-8770-d7f099683474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:37.634387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ceb55-51a6-41bf-8770-d7f099683474 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:37.640040 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ceb55-51a6-41bf-8770-d7f099683474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:37.640040 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ceb55-51a6-41bf-8770-d7f099683474 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:37.640040 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ceb55-51a6-41bf-8770-d7f099683474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:37.640040 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226ceb55-51a6-41bf-8770-d7f099683474 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:37.724841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c4471-4b73-49fe-97a8-116da4a81cf0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:37.727535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2e1c4471-4b73-49fe-97a8-116da4a81cf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:37.750765 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1c4471-4b73-49fe-97a8-116da4a81cf0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:37.760997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd852a1-6185-43b7-b13e-f155a16d4b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:37.762962 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd852a1-6185-43b7-b13e-f155a16d4b96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:37.763322 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd852a1-6185-43b7-b13e-f155a16d4b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:37.776089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd852a1-6185-43b7-b13e-f155a16d4b96 admin admin] PATH_INFO: `/v3/projects/ffdbc5fb68b0454d9dee49665b736efd` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:37.779128 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd852a1-6185-43b7-b13e-f155a16d4b96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffdbc5fb68b0454d9dee49665b736efd)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:37.783039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd852a1-6185-43b7-b13e-f155a16d4b96 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:37.797069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 879/1765] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:37 2022] POST /identity/v3/users => generated 294 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:37.801917 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ceb55-51a6-41bf-8770-d7f099683474 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:37.816448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226ceb55-51a6-41bf-8770-d7f099683474 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:37.822802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b5b4a-0926-40b3-9977-b9f5a039765f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:37.823716 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b5b4a-0926-40b3-9977-b9f5a039765f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:37.824027 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b5b4a-0926-40b3-9977-b9f5a039765f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:37.824385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b5b4a-0926-40b3-9977-b9f5a039765f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:37.825111 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b5b4a-0926-40b3-9977-b9f5a039765f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:37.827013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b5b4a-0926-40b3-9977-b9f5a039765f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:37.840403 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 880/1766] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:37.844293 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b5b4a-0926-40b3-9977-b9f5a039765f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:37.857105 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b5b4a-0926-40b3-9977-b9f5a039765f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:37.861864 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e584c-b3e3-4e4e-b64d-4300a62ccd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:37.862474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e584c-b3e3-4e4e-b64d-4300a62ccd01 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:37.862757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e584c-b3e3-4e4e-b64d-4300a62ccd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:37.863031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e584c-b3e3-4e4e-b64d-4300a62ccd01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:37.863699 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e584c-b3e3-4e4e-b64d-4300a62ccd01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:37.865305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e584c-b3e3-4e4e-b64d-4300a62ccd01 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:37.875570 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 881/1767] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:37.879831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e584c-b3e3-4e4e-b64d-4300a62ccd01 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:37.890891 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e584c-b3e3-4e4e-b64d-4300a62ccd01 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:37.898201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad50247-20b7-42aa-815d-45a89725b5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:37.899073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad50247-20b7-42aa-815d-45a89725b5e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:37.899356 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad50247-20b7-42aa-815d-45a89725b5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:37.899629 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad50247-20b7-42aa-815d-45a89725b5e1 admin admin] PATH_INFO: `/v3/projects/7485987bf80d42f1bc4991d3b22fc845/users/852aafdae5434fba9683c21154151cf4/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:37.915058 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad50247-20b7-42aa-815d-45a89725b5e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7485987bf80d42f1bc4991d3b22fc845, user_id=852aafdae5434fba9683c21154151cf4, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:37.917718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad50247-20b7-42aa-815d-45a89725b5e1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:38.023232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:38.133426 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 882/1768] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:37 2022] PUT /identity/v3/projects/7485987bf80d42f1bc4991d3b22fc845/users/852aafdae5434fba9683c21154151cf4/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:38.148094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:38.148810 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d0c19-6f7d-47f3-ba60-2a13c6ec622e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:38.149099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d0c19-6f7d-47f3-ba60-2a13c6ec622e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:38.149378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d0c19-6f7d-47f3-ba60-2a13c6ec622e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:38.149996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-abd852a1-6185-43b7-b13e-f155a16d4b96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffdbc5fb68b0454d9dee49665b736efd', 'request_id': 'req-abd852a1-6185-43b7-b13e-f155a16d4b96'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:38.157220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-abd852a1-6185-43b7-b13e-f155a16d4b96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffdbc5fb68b0454d9dee49665b736efd', 'request_id': 'req-abd852a1-6185-43b7-b13e-f155a16d4b96'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:38.177795 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-360d0c19-6f7d-47f3-ba60-2a13c6ec622e None None] MFA Rules not processed for user `852aafdae5434fba9683c21154151cf4`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:38.180029 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-360d0c19-6f7d-47f3-ba60-2a13c6ec622e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:38.243948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 883/1769] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:38.248215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d0c19-6f7d-47f3-ba60-2a13c6ec622e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:38.269187 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-abd852a1-6185-43b7-b13e-f155a16d4b96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffdbc5fb68b0454d9dee49665b736efd', 'request_id': 'req-abd852a1-6185-43b7-b13e-f155a16d4b96'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:38.282204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-abd852a1-6185-43b7-b13e-f155a16d4b96 admin admin] The token cache is being invalidate because project ffdbc5fb68b0454d9dee49665b736efd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:03:38.283833 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-abd852a1-6185-43b7-b13e-f155a16d4b96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:38.287601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-abd852a1-6185-43b7-b13e-f155a16d4b96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:38.290205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 887/1770] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:03:37 2022] DELETE /identity/v3/projects/ffdbc5fb68b0454d9dee49665b736efd => generated 0 bytes in 566 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:38.296883 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012dda8f-7749-44a5-86ea-45e7caef62dc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:38.297366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012dda8f-7749-44a5-86ea-45e7caef62dc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:38.297854 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012dda8f-7749-44a5-86ea-45e7caef62dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:38.319055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d0c19-6f7d-47f3-ba60-2a13c6ec622e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:38.322398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-012dda8f-7749-44a5-86ea-45e7caef62dc None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:38.323624 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-012dda8f-7749-44a5-86ea-45e7caef62dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:38.325865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7670f3d-adc3-44bb-be4a-090a779743d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:38.326969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7670f3d-adc3-44bb-be4a-090a779743d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:38.327442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7670f3d-adc3-44bb-be4a-090a779743d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:38.327995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7670f3d-adc3-44bb-be4a-090a779743d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:38.329335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7670f3d-adc3-44bb-be4a-090a779743d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:38.335907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7670f3d-adc3-44bb-be4a-090a779743d7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:38.343065 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 888/1771] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:38.344581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe2d53-6b99-43bf-809b-2fcd00c39d6b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:38.344867 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe2d53-6b99-43bf-809b-2fcd00c39d6b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:38.345146 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe2d53-6b99-43bf-809b-2fcd00c39d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:38.374187 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-22fe2d53-6b99-43bf-809b-2fcd00c39d6b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:38.375542 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-22fe2d53-6b99-43bf-809b-2fcd00c39d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:38.386505 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 889/1772] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:38.390729 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe2d53-6b99-43bf-809b-2fcd00c39d6b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:38.403591 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe2d53-6b99-43bf-809b-2fcd00c39d6b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:38.409229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b19561-f6f4-4e64-83af-f5e619b49a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:38.410568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b19561-f6f4-4e64-83af-f5e619b49a0c admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:38.410994 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b19561-f6f4-4e64-83af-f5e619b49a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:38.411547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b19561-f6f4-4e64-83af-f5e619b49a0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:38.418276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b19561-f6f4-4e64-83af-f5e619b49a0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:38.418276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b19561-f6f4-4e64-83af-f5e619b49a0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:38.418276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b19561-f6f4-4e64-83af-f5e619b49a0c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:38.430476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 890/1773] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:03:38.435359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b19561-f6f4-4e64-83af-f5e619b49a0c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:38.448124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b19561-f6f4-4e64-83af-f5e619b49a0c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:38.456078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed29263-75de-4896-9a95-88812e88c074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:38.457056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed29263-75de-4896-9a95-88812e88c074 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:38.457347 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed29263-75de-4896-9a95-88812e88c074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:38.457640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed29263-75de-4896-9a95-88812e88c074 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:38.458494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed29263-75de-4896-9a95-88812e88c074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:38.460698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 884/1774] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:38 2022] POST /identity/v3/projects => generated 387 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:03:38.464658 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed29263-75de-4896-9a95-88812e88c074 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:38.469801 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7670f3d-adc3-44bb-be4a-090a779743d7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:38.472630 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a7670f3d-adc3-44bb-be4a-090a779743d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:38.557457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7670f3d-adc3-44bb-be4a-090a779743d7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:38.562657 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5eda94-5134-4978-bd5c-2873c58684e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:38.563373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5eda94-5134-4978-bd5c-2873c58684e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:38.563674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5eda94-5134-4978-bd5c-2873c58684e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:38.563955 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5eda94-5134-4978-bd5c-2873c58684e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:38.564658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5eda94-5134-4978-bd5c-2873c58684e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:38.566623 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5eda94-5134-4978-bd5c-2873c58684e0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:38.679622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 891/1775] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:38 2022] POST /identity/v3/projects => generated 381 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:03:38.680799 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed29263-75de-4896-9a95-88812e88c074 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:38.742075 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 885/1776] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:38 2022] POST /identity/v3/users => generated 300 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:03:38.745792 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5eda94-5134-4978-bd5c-2873c58684e0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:38.756108 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5eda94-5134-4978-bd5c-2873c58684e0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:38.758443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed29263-75de-4896-9a95-88812e88c074 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:38.761871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c778c-570a-46aa-be1b-d47faf00b5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:38.762566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c778c-570a-46aa-be1b-d47faf00b5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:38.762917 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c778c-570a-46aa-be1b-d47faf00b5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:38.763260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c778c-570a-46aa-be1b-d47faf00b5e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:38.763755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795698ac-8384-465a-b4ea-3ed44d7aa00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:38.764113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c778c-570a-46aa-be1b-d47faf00b5e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:38.766791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714c778c-570a-46aa-be1b-d47faf00b5e9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:38.766936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795698ac-8384-465a-b4ea-3ed44d7aa00f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:38.766936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795698ac-8384-465a-b4ea-3ed44d7aa00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:38.767211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795698ac-8384-465a-b4ea-3ed44d7aa00f admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:38.767828 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795698ac-8384-465a-b4ea-3ed44d7aa00f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:38.769968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795698ac-8384-465a-b4ea-3ed44d7aa00f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:38.783394 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 886/1777] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:03:38.787450 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c778c-570a-46aa-be1b-d47faf00b5e9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:38.813830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714c778c-570a-46aa-be1b-d47faf00b5e9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:38.827261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a9f81-49cb-45b1-9da7-7d99c4910ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:38.828060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a9f81-49cb-45b1-9da7-7d99c4910ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:38.828546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a9f81-49cb-45b1-9da7-7d99c4910ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:38.828829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a9f81-49cb-45b1-9da7-7d99c4910ac5 admin admin] PATH_INFO: `/v3/projects/95b86810df6048b7b95b05363197826f/users/affa98b44a394d28b595bd47d0957550/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:38.848954 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a9f81-49cb-45b1-9da7-7d99c4910ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95b86810df6048b7b95b05363197826f, user_id=affa98b44a394d28b595bd47d0957550, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:38.853280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a9f81-49cb-45b1-9da7-7d99c4910ac5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:38.888909 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 892/1778] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:38 2022] POST /identity/v3/users => generated 291 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:03:38.891388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795698ac-8384-465a-b4ea-3ed44d7aa00f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:38.908988 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795698ac-8384-465a-b4ea-3ed44d7aa00f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:38.924846 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc6d3c-6821-4e14-839b-79c2ea8cf7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:38.925557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc6d3c-6821-4e14-839b-79c2ea8cf7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:38.926103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc6d3c-6821-4e14-839b-79c2ea8cf7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:38.926391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcc6d3c-6821-4e14-839b-79c2ea8cf7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:38.927351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc6d3c-6821-4e14-839b-79c2ea8cf7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:38.929367 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcc6d3c-6821-4e14-839b-79c2ea8cf7f0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:38.942977 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 893/1779] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:38.947617 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc6d3c-6821-4e14-839b-79c2ea8cf7f0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:38.992535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcc6d3c-6821-4e14-839b-79c2ea8cf7f0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:38.999007 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657540c0-75a5-4315-ace0-e4daf871be66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:38.999628 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657540c0-75a5-4315-ace0-e4daf871be66 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:38.999946 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657540c0-75a5-4315-ace0-e4daf871be66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:39.000220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657540c0-75a5-4315-ace0-e4daf871be66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:39.000858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657540c0-75a5-4315-ace0-e4daf871be66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:39.002510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657540c0-75a5-4315-ace0-e4daf871be66 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:39.015914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 894/1780] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:39.032668 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657540c0-75a5-4315-ace0-e4daf871be66 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:39.048470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 887/1781] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:38 2022] PUT /identity/v3/projects/95b86810df6048b7b95b05363197826f/users/affa98b44a394d28b595bd47d0957550/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:39.049951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657540c0-75a5-4315-ace0-e4daf871be66 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:39.054399 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a9f81-49cb-45b1-9da7-7d99c4910ac5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:39.164778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb38c08-a500-41cc-930a-bf4ea0ab3d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:39.165696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb38c08-a500-41cc-930a-bf4ea0ab3d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:39.166197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb38c08-a500-41cc-930a-bf4ea0ab3d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:39.166658 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb38c08-a500-41cc-930a-bf4ea0ab3d28 admin admin] PATH_INFO: `/v3/projects/1ce539f13fe8493fa0fe48620d9e1dea/users/152105c81a1a4c989d41f9a0fbc6d68c/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:39.188666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb38c08-a500-41cc-930a-bf4ea0ab3d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce539f13fe8493fa0fe48620d9e1dea, user_id=152105c81a1a4c989d41f9a0fbc6d68c, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:39.191530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb38c08-a500-41cc-930a-bf4ea0ab3d28 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:39.202546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a9f81-49cb-45b1-9da7-7d99c4910ac5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:39.207346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76aefc-6688-499a-a67e-48a6baa08e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:39.208323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76aefc-6688-499a-a67e-48a6baa08e5a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:39.208323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76aefc-6688-499a-a67e-48a6baa08e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:39.208703 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76aefc-6688-499a-a67e-48a6baa08e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:39.209187 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76aefc-6688-499a-a67e-48a6baa08e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:39.211042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a76aefc-6688-499a-a67e-48a6baa08e5a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:39.224503 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 888/1782] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:03:39.225926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76aefc-6688-499a-a67e-48a6baa08e5a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:39.227710 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5a76aefc-6688-499a-a67e-48a6baa08e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:39.245411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76aefc-6688-499a-a67e-48a6baa08e5a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:39.252545 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02ac05-2e82-4a77-a490-c46bba8dbf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:39.253196 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02ac05-2e82-4a77-a490-c46bba8dbf6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:39.258301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02ac05-2e82-4a77-a490-c46bba8dbf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:39.258301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d02ac05-2e82-4a77-a490-c46bba8dbf6d admin admin] PATH_INFO: `/v3/users/b06193cf39004cc9b1fa37f6bf9de115` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:39.258301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02ac05-2e82-4a77-a490-c46bba8dbf6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b06193cf39004cc9b1fa37f6bf9de115)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:39.258301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d02ac05-2e82-4a77-a490-c46bba8dbf6d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:39.349687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 895/1783] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:39 2022] PUT /identity/v3/projects/1ce539f13fe8493fa0fe48620d9e1dea/users/152105c81a1a4c989d41f9a0fbc6d68c/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:39.362558 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb38c08-a500-41cc-930a-bf4ea0ab3d28 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:39.489986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb38c08-a500-41cc-930a-bf4ea0ab3d28 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:39.497419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bde06-5c63-43c7-9a10-c60c29250b53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:39.498435 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bde06-5c63-43c7-9a10-c60c29250b53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:39.498837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bde06-5c63-43c7-9a10-c60c29250b53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:39.499261 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bde06-5c63-43c7-9a10-c60c29250b53 admin admin] PATH_INFO: `/v3/projects/95b86810df6048b7b95b05363197826f/users/affa98b44a394d28b595bd47d0957550/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:39.504871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bde06-5c63-43c7-9a10-c60c29250b53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95b86810df6048b7b95b05363197826f, user_id=affa98b44a394d28b595bd47d0957550, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:39.514056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bde06-5c63-43c7-9a10-c60c29250b53 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:39.530371 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 896/1784] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:39 2022] PUT /identity/v3/projects/95b86810df6048b7b95b05363197826f/users/affa98b44a394d28b595bd47d0957550/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:39.533294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc759c45-9b13-477f-9e7d-18156e590b0e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:39.534238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc759c45-9b13-477f-9e7d-18156e590b0e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:39.534396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc759c45-9b13-477f-9e7d-18156e590b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:39.567286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8d02ac05-2e82-4a77-a490-c46bba8dbf6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b06193cf39004cc9b1fa37f6bf9de115'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:39.592645 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-bc759c45-9b13-477f-9e7d-18156e590b0e None None] MFA Rules not processed for user `152105c81a1a4c989d41f9a0fbc6d68c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:39.593903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bc759c45-9b13-477f-9e7d-18156e590b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:39.683153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 897/1785] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:39.689134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fec79f-9816-4440-bf77-8a0600776f57 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:39.689134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fec79f-9816-4440-bf77-8a0600776f57 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:39.689134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fec79f-9816-4440-bf77-8a0600776f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:39.727987 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8d02ac05-2e82-4a77-a490-c46bba8dbf6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b06193cf39004cc9b1fa37f6bf9de115'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:39.729766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8d02ac05-2e82-4a77-a490-c46bba8dbf6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b06193cf39004cc9b1fa37f6bf9de115'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:39.731898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8d02ac05-2e82-4a77-a490-c46bba8dbf6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b06193cf39004cc9b1fa37f6bf9de115'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:39.738436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-98fec79f-9816-4440-bf77-8a0600776f57 None None] MFA Rules not processed for user `affa98b44a394d28b595bd47d0957550`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:39.743734 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-98fec79f-9816-4440-bf77-8a0600776f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:39.754768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8d02ac05-2e82-4a77-a490-c46bba8dbf6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b06193cf39004cc9b1fa37f6bf9de115'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:39.778108 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 889/1786] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:03:39 2022] DELETE /identity/v3/users/b06193cf39004cc9b1fa37f6bf9de115 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:39.778108 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02ac05-2e82-4a77-a490-c46bba8dbf6d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:39.778108 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8d02ac05-2e82-4a77-a490-c46bba8dbf6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:39.860121 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 898/1787] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:39.861635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33951021-caf8-4ea7-a643-85388b3e9ae2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:39.861925 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33951021-caf8-4ea7-a643-85388b3e9ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:39.862209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33951021-caf8-4ea7-a643-85388b3e9ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:39.877292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d02ac05-2e82-4a77-a490-c46bba8dbf6d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:39.882652 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36362ad-47ab-45b9-9457-8fec90254ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:39.883275 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36362ad-47ab-45b9-9457-8fec90254ef1 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:39.883384 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-33951021-caf8-4ea7-a643-85388b3e9ae2 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:39.883561 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36362ad-47ab-45b9-9457-8fec90254ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:39.883917 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36362ad-47ab-45b9-9457-8fec90254ef1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:39.884559 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36362ad-47ab-45b9-9457-8fec90254ef1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:39.884666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-33951021-caf8-4ea7-a643-85388b3e9ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:39.886840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36362ad-47ab-45b9-9457-8fec90254ef1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:39.893288 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 899/1788] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:39.894131 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33951021-caf8-4ea7-a643-85388b3e9ae2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:39.895897 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-33951021-caf8-4ea7-a643-85388b3e9ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:39.968412 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33951021-caf8-4ea7-a643-85388b3e9ae2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:39.973098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be10a1-0587-416c-a90c-25e243d1bd1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:39.973904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be10a1-0587-416c-a90c-25e243d1bd1d service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:39.974171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be10a1-0587-416c-a90c-25e243d1bd1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:39.974476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56be10a1-0587-416c-a90c-25e243d1bd1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:40.051397 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be10a1-0587-416c-a90c-25e243d1bd1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:40.052901 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56be10a1-0587-416c-a90c-25e243d1bd1d service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:40.063767 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 900/1789] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:40.065072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be10a1-0587-416c-a90c-25e243d1bd1d service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:40.066733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-56be10a1-0587-416c-a90c-25e243d1bd1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:40.109418 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 890/1790] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:39 2022] POST /identity/v3/projects => generated 383 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:40.110554 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36362ad-47ab-45b9-9457-8fec90254ef1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:40.119301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d36362ad-47ab-45b9-9457-8fec90254ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:40.196900 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56be10a1-0587-416c-a90c-25e243d1bd1d service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:40.290895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36362ad-47ab-45b9-9457-8fec90254ef1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:40.298485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b23af7-ae09-48a6-86b4-d71effb79e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:40.300331 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b23af7-ae09-48a6-86b4-d71effb79e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:40.300900 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b23af7-ae09-48a6-86b4-d71effb79e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:40.301114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b23af7-ae09-48a6-86b4-d71effb79e07 admin admin] PATH_INFO: `/v3/users/852aafdae5434fba9683c21154151cf4` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:40.302907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b23af7-ae09-48a6-86b4-d71effb79e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=852aafdae5434fba9683c21154151cf4)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:40.306855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b23af7-ae09-48a6-86b4-d71effb79e07 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:40.307827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9fbf2-6ef3-4cc4-871d-331174101011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:40.308965 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9fbf2-6ef3-4cc4-871d-331174101011 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:40.309118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9fbf2-6ef3-4cc4-871d-331174101011 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:40.309607 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a9fbf2-6ef3-4cc4-871d-331174101011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:40.324871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9fbf2-6ef3-4cc4-871d-331174101011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:40.327202 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a9fbf2-6ef3-4cc4-871d-331174101011 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:40.340106 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 901/1791] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:40.341026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9fbf2-6ef3-4cc4-871d-331174101011 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:40.351882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a9fbf2-6ef3-4cc4-871d-331174101011 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:40.356691 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a7cec-6713-420b-bc40-225f908a84b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:40.357299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a7cec-6713-420b-bc40-225f908a84b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:40.357576 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a7cec-6713-420b-bc40-225f908a84b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:40.357933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a7cec-6713-420b-bc40-225f908a84b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:40.358543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a7cec-6713-420b-bc40-225f908a84b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:40.360497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7a7cec-6713-420b-bc40-225f908a84b2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:40.545160 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 902/1792] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:40 2022] POST /identity/v3/users => generated 298 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:40.549685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a7cec-6713-420b-bc40-225f908a84b2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:40.560691 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a7cec-6713-420b-bc40-225f908a84b2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:40.566352 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf8ffd-abbe-46fe-96f3-931b6e1019ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:40.567493 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf8ffd-abbe-46fe-96f3-931b6e1019ec admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:40.567975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf8ffd-abbe-46fe-96f3-931b6e1019ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:40.569973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf8ffd-abbe-46fe-96f3-931b6e1019ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:40.570242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf8ffd-abbe-46fe-96f3-931b6e1019ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:40.572632 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf8ffd-abbe-46fe-96f3-931b6e1019ec admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:40.591403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 903/1793] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:03:40.595902 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf8ffd-abbe-46fe-96f3-931b6e1019ec admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:40.607632 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf8ffd-abbe-46fe-96f3-931b6e1019ec admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:40.617952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65123cff-7c26-4d2f-ad47-0d33a251acb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:40.618747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65123cff-7c26-4d2f-ad47-0d33a251acb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:40.619134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65123cff-7c26-4d2f-ad47-0d33a251acb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:40.619525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65123cff-7c26-4d2f-ad47-0d33a251acb1 admin admin] PATH_INFO: `/v3/projects/8858ae8903bf45018b353a2f3948e963/users/574859147d14417cb5f9fddfbbace502/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:40.638004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65123cff-7c26-4d2f-ad47-0d33a251acb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8858ae8903bf45018b353a2f3948e963, user_id=574859147d14417cb5f9fddfbbace502, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:40.640875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65123cff-7c26-4d2f-ad47-0d33a251acb1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:40.796686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-10b23af7-ae09-48a6-86b4-d71effb79e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '852aafdae5434fba9683c21154151cf4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:41.046395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-10b23af7-ae09-48a6-86b4-d71effb79e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '852aafdae5434fba9683c21154151cf4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:41.047090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-10b23af7-ae09-48a6-86b4-d71effb79e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '852aafdae5434fba9683c21154151cf4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:41.047747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-10b23af7-ae09-48a6-86b4-d71effb79e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '852aafdae5434fba9683c21154151cf4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:41.052565 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 904/1794] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:40 2022] PUT /identity/v3/projects/8858ae8903bf45018b353a2f3948e963/users/574859147d14417cb5f9fddfbbace502/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:41.057289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65123cff-7c26-4d2f-ad47-0d33a251acb1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:41.060199 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-65123cff-7c26-4d2f-ad47-0d33a251acb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:41.072696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-10b23af7-ae09-48a6-86b4-d71effb79e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '852aafdae5434fba9683c21154151cf4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:41.115386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 891/1795] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:03:40 2022] DELETE /identity/v3/users/852aafdae5434fba9683c21154151cf4 => generated 0 bytes in 1006 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:41.120302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b23af7-ae09-48a6-86b4-d71effb79e07 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:41.124049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-10b23af7-ae09-48a6-86b4-d71effb79e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:41.172310 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65123cff-7c26-4d2f-ad47-0d33a251acb1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:41.239466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b23af7-ae09-48a6-86b4-d71effb79e07 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:41.245378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:41.246101 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:41.246454 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:41.246804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d admin admin] PATH_INFO: `/v3/projects/991f7c77d7734c56a3a22458ce7dc601` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:41.249573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=991f7c77d7734c56a3a22458ce7dc601)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:41.253869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:41.274648 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373810a4-4fe4-4a81-b09c-e38732262eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:41.275754 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373810a4-4fe4-4a81-b09c-e38732262eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:41.276536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373810a4-4fe4-4a81-b09c-e38732262eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:41.277012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373810a4-4fe4-4a81-b09c-e38732262eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:41.278137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373810a4-4fe4-4a81-b09c-e38732262eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:41.282855 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373810a4-4fe4-4a81-b09c-e38732262eb2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:41.302770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 905/1796] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:41 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:41.305362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b6553-8bc2-4957-8dab-b2ae0f15c313 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:41.308557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b6553-8bc2-4957-8dab-b2ae0f15c313 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:41.308557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b6553-8bc2-4957-8dab-b2ae0f15c313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:41.332708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-121b6553-8bc2-4957-8dab-b2ae0f15c313 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:41.334543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-121b6553-8bc2-4957-8dab-b2ae0f15c313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:41.346388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 906/1797] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:41.347824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b6553-8bc2-4957-8dab-b2ae0f15c313 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:41.359451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b6553-8bc2-4957-8dab-b2ae0f15c313 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:41.365230 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34c528-fa6d-4f8b-b49c-96d9eb08515a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:41.366745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34c528-fa6d-4f8b-b49c-96d9eb08515a admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:41.367197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34c528-fa6d-4f8b-b49c-96d9eb08515a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:41.367578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d34c528-fa6d-4f8b-b49c-96d9eb08515a admin admin] PATH_INFO: `/v3/projects/8858ae8903bf45018b353a2f3948e963/users/574859147d14417cb5f9fddfbbace502/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:41.369712 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34c528-fa6d-4f8b-b49c-96d9eb08515a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8858ae8903bf45018b353a2f3948e963, user_id=574859147d14417cb5f9fddfbbace502, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:41.372816 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d34c528-fa6d-4f8b-b49c-96d9eb08515a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:41.386919 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 907/1798] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:41 2022] PUT /identity/v3/projects/8858ae8903bf45018b353a2f3948e963/users/574859147d14417cb5f9fddfbbace502/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:41.388006 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34c528-fa6d-4f8b-b49c-96d9eb08515a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:41.389832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1d34c528-fa6d-4f8b-b49c-96d9eb08515a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:41.466514 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d34c528-fa6d-4f8b-b49c-96d9eb08515a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:41.470135 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff59a6-5fd2-4721-b1fd-baa1c99fa2c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:41.470740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff59a6-5fd2-4721-b1fd-baa1c99fa2c6 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:41.471021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff59a6-5fd2-4721-b1fd-baa1c99fa2c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:41.471324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff59a6-5fd2-4721-b1fd-baa1c99fa2c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:41.506107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '991f7c77d7734c56a3a22458ce7dc601', 'request_id': 'req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:41.521935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '991f7c77d7734c56a3a22458ce7dc601', 'request_id': 'req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:41.546167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff59a6-5fd2-4721-b1fd-baa1c99fa2c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:41.547815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff59a6-5fd2-4721-b1fd-baa1c99fa2c6 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:41.560496 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 908/1799] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:41.561778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47e3d5-9924-44d9-a4d6-18b4c9ce0ae5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:41.562063 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47e3d5-9924-44d9-a4d6-18b4c9ce0ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:41.562336 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47e3d5-9924-44d9-a4d6-18b4c9ce0ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:41.595875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1a47e3d5-9924-44d9-a4d6-18b4c9ce0ae5 None None] MFA Rules not processed for user `574859147d14417cb5f9fddfbbace502`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:41.597093 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1a47e3d5-9924-44d9-a4d6-18b4c9ce0ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:41.705521 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 909/1800] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:41.717136 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47e3d5-9924-44d9-a4d6-18b4c9ce0ae5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:41.718612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1a47e3d5-9924-44d9-a4d6-18b4c9ce0ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:41.771699 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '991f7c77d7734c56a3a22458ce7dc601', 'request_id': 'req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:41.785947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d admin admin] The token cache is being invalidate because project 991f7c77d7734c56a3a22458ce7dc601 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:03:41.786290 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:41.787432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:41.792282 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 892/1801] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:03:41 2022] DELETE /identity/v3/projects/991f7c77d7734c56a3a22458ce7dc601 => generated 0 bytes in 676 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:41.797207 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:41.800803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47e3d5-9924-44d9-a4d6-18b4c9ce0ae5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:41.804307 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:41.818576 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d1f96-8591-497e-8b89-0145fa74bba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:41.821684 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d1f96-8591-497e-8b89-0145fa74bba4 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:41.822181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d1f96-8591-497e-8b89-0145fa74bba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:41.822536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d1f96-8591-497e-8b89-0145fa74bba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:41.825473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5e1d1f96-8591-497e-8b89-0145fa74bba4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:41.855628 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d1f96-8591-497e-8b89-0145fa74bba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:41.857362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1d1f96-8591-497e-8b89-0145fa74bba4 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:41.874719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 910/1802] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:03:41.878400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3000f-f9be-4890-8436-2d09c46b560f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:41.882709 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3000f-f9be-4890-8436-2d09c46b560f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:41.882709 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3000f-f9be-4890-8436-2d09c46b560f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:41.962192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca4cac-63f7-4c44-ba24-2f620655bf6d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:41.990624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaabe8-17a5-4f82-8d57-5894c28d182a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:41.991938 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaabe8-17a5-4f82-8d57-5894c28d182a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:41.992236 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaabe8-17a5-4f82-8d57-5894c28d182a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:41.992519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaabe8-17a5-4f82-8d57-5894c28d182a admin admin] PATH_INFO: `/v3/users/7c04a0b5622d4078bd088566e4a0fd9c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:42.004831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaabe8-17a5-4f82-8d57-5894c28d182a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c04a0b5622d4078bd088566e4a0fd9c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:42.005248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaabe8-17a5-4f82-8d57-5894c28d182a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:42.008011 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-37f3000f-f9be-4890-8436-2d09c46b560f None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:42.009537 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-37f3000f-f9be-4890-8436-2d09c46b560f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:42.167099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 911/1803] 10.222.0.67 () {58 vars in 1009 bytes} [Mon Sep 26 18:03:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:42.191769 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3000f-f9be-4890-8436-2d09c46b560f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:42.194162 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-37f3000f-f9be-4890-8436-2d09c46b560f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:42.194900 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:42.211420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3000f-f9be-4890-8436-2d09c46b560f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:42.217000 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ef659-0f94-4780-ad20-4721b13c0487 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:42.217709 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ef659-0f94-4780-ad20-4721b13c0487 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:42.217994 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ef659-0f94-4780-ad20-4721b13c0487 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:42.218265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934ef659-0f94-4780-ad20-4721b13c0487 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:42.220196 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-934ef659-0f94-4780-ad20-4721b13c0487 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:42.308142 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ef659-0f94-4780-ad20-4721b13c0487 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:42.309723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934ef659-0f94-4780-ad20-4721b13c0487 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:42.327419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 912/1804] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:03:42.327419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:42.362401 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ef659-0f94-4780-ad20-4721b13c0487 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:42.364785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-934ef659-0f94-4780-ad20-4721b13c0487 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:42.394845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bbbaabe8-17a5-4f82-8d57-5894c28d182a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c04a0b5622d4078bd088566e4a0fd9c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:42.404473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934ef659-0f94-4780-ad20-4721b13c0487 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:42.412338 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:42.413299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:42.413583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:42.413851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7 admin admin] PATH_INFO: `/v3/projects/7485987bf80d42f1bc4991d3b22fc845` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:42.415051 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7485987bf80d42f1bc4991d3b22fc845)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:42.417301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:42.610756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bbbaabe8-17a5-4f82-8d57-5894c28d182a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c04a0b5622d4078bd088566e4a0fd9c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:42.611780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bbbaabe8-17a5-4f82-8d57-5894c28d182a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c04a0b5622d4078bd088566e4a0fd9c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:42.612620 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bbbaabe8-17a5-4f82-8d57-5894c28d182a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c04a0b5622d4078bd088566e4a0fd9c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:42.644635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bbbaabe8-17a5-4f82-8d57-5894c28d182a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c04a0b5622d4078bd088566e4a0fd9c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:42.663482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7485987bf80d42f1bc4991d3b22fc845', 'request_id': 'req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:42.670602 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 893/1805] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:03:41 2022] DELETE /identity/v3/users/7c04a0b5622d4078bd088566e4a0fd9c => generated 0 bytes in 874 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:42.674314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7485987bf80d42f1bc4991d3b22fc845', 'request_id': 'req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:42.717755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaabe8-17a5-4f82-8d57-5894c28d182a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:42.725149 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bbbaabe8-17a5-4f82-8d57-5894c28d182a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:42.820851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7485987bf80d42f1bc4991d3b22fc845', 'request_id': 'req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:42.834408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7 admin admin] The token cache is being invalidate because project 7485987bf80d42f1bc4991d3b22fc845 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:03:42.834900 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:42.836424 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:42.842033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 913/1806] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:03:42 2022] DELETE /identity/v3/projects/7485987bf80d42f1bc4991d3b22fc845 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:42.844254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:42.845895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:42.969185 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ceb9c-d1f9-4ef0-84d1-daaccf46a0c7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:43.005132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaabe8-17a5-4f82-8d57-5894c28d182a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:43.013754 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d6b2a-512b-4862-a446-c7a6041cc7be service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:43.015013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d6b2a-512b-4862-a446-c7a6041cc7be service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:43.015686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d6b2a-512b-4862-a446-c7a6041cc7be service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:43.016269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d6b2a-512b-4862-a446-c7a6041cc7be service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:43.020218 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a58d6b2a-512b-4862-a446-c7a6041cc7be service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:43.183630 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce05a007-906f-437e-b056-667ecac29d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:43.184530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05a007-906f-437e-b056-667ecac29d20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:43.184983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05a007-906f-437e-b056-667ecac29d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:43.185273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce05a007-906f-437e-b056-667ecac29d20 admin admin] PATH_INFO: `/v3/users/affa98b44a394d28b595bd47d0957550` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:43.187048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05a007-906f-437e-b056-667ecac29d20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=affa98b44a394d28b595bd47d0957550)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:43.189142 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce05a007-906f-437e-b056-667ecac29d20 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:43.246362 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d6b2a-512b-4862-a446-c7a6041cc7be service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:43.249134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d6b2a-512b-4862-a446-c7a6041cc7be service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:43.264606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 894/1807] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:03:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:03:43.513766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d6b2a-512b-4862-a446-c7a6041cc7be service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:43.515498 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a58d6b2a-512b-4862-a446-c7a6041cc7be service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:43.526982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ce05a007-906f-437e-b056-667ecac29d20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'affa98b44a394d28b595bd47d0957550'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:43.550262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d6b2a-512b-4862-a446-c7a6041cc7be service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:43.565602 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a39754-b17a-4775-bd07-f79ecadd083b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:43.580190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a39754-b17a-4775-bd07-f79ecadd083b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:43.580190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a39754-b17a-4775-bd07-f79ecadd083b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:43.580190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a39754-b17a-4775-bd07-f79ecadd083b admin admin] PATH_INFO: `/v3/projects/5bf22dee0e274298b7991d8d89241a8d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:43.580190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a39754-b17a-4775-bd07-f79ecadd083b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bf22dee0e274298b7991d8d89241a8d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:43.586082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a39754-b17a-4775-bd07-f79ecadd083b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:43.697190 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ce05a007-906f-437e-b056-667ecac29d20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'affa98b44a394d28b595bd47d0957550'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:43.697817 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ce05a007-906f-437e-b056-667ecac29d20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'affa98b44a394d28b595bd47d0957550'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:43.698449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ce05a007-906f-437e-b056-667ecac29d20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'affa98b44a394d28b595bd47d0957550'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:43.706434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ce05a007-906f-437e-b056-667ecac29d20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'affa98b44a394d28b595bd47d0957550'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:43.727018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 914/1808] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:03:42 2022] DELETE /identity/v3/users/affa98b44a394d28b595bd47d0957550 => generated 0 bytes in 884 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:43.752576 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:43.797686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-76a39754-b17a-4775-bd07-f79ecadd083b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bf22dee0e274298b7991d8d89241a8d', 'request_id': 'req-76a39754-b17a-4775-bd07-f79ecadd083b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:43.808980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-76a39754-b17a-4775-bd07-f79ecadd083b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bf22dee0e274298b7991d8d89241a8d', 'request_id': 'req-76a39754-b17a-4775-bd07-f79ecadd083b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:44.019190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-76a39754-b17a-4775-bd07-f79ecadd083b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bf22dee0e274298b7991d8d89241a8d', 'request_id': 'req-76a39754-b17a-4775-bd07-f79ecadd083b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:44.043831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-76a39754-b17a-4775-bd07-f79ecadd083b admin admin] The token cache is being invalidate because project 5bf22dee0e274298b7991d8d89241a8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:03:44.044224 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-76a39754-b17a-4775-bd07-f79ecadd083b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:44.044986 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-76a39754-b17a-4775-bd07-f79ecadd083b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:44.046592 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 895/1809] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:03:43 2022] DELETE /identity/v3/projects/5bf22dee0e274298b7991d8d89241a8d => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:44.053688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b8f83-c715-4f8a-bf03-faadf5933a41 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:44.054123 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b8f83-c715-4f8a-bf03-faadf5933a41 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:44.054393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3b8f83-c715-4f8a-bf03-faadf5933a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:44.098666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7d3b8f83-c715-4f8a-bf03-faadf5933a41 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:44.100729 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7d3b8f83-c715-4f8a-bf03-faadf5933a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:44.207582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 915/1810] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:44.208598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00952ec5-aec1-4d4c-a426-1dc235bb1132 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:44.208598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00952ec5-aec1-4d4c-a426-1dc235bb1132 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:44.208598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00952ec5-aec1-4d4c-a426-1dc235bb1132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:44.253133 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-00952ec5-aec1-4d4c-a426-1dc235bb1132 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:44.255679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-00952ec5-aec1-4d4c-a426-1dc235bb1132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:44.268149 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 896/1811] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:44.270089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b8f83-c715-4f8a-bf03-faadf5933a41 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:44.285900 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3b8f83-c715-4f8a-bf03-faadf5933a41 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:44.308858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c2a34-e325-4970-973e-6491e2479289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:44.311445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c2a34-e325-4970-973e-6491e2479289 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:44.312252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c2a34-e325-4970-973e-6491e2479289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:44.312927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240c2a34-e325-4970-973e-6491e2479289 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:44.313813 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c2a34-e325-4970-973e-6491e2479289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:44.314577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c2a34-e325-4970-973e-6491e2479289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:44.318077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240c2a34-e325-4970-973e-6491e2479289 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:44.338951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 916/1812] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:03:44.344584 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00952ec5-aec1-4d4c-a426-1dc235bb1132 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:44.359311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00952ec5-aec1-4d4c-a426-1dc235bb1132 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:44.366063 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db85e41-8ac6-475a-ae19-f530dff13712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:44.366845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db85e41-8ac6-475a-ae19-f530dff13712 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:44.367201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db85e41-8ac6-475a-ae19-f530dff13712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:44.367546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db85e41-8ac6-475a-ae19-f530dff13712 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:44.368303 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db85e41-8ac6-475a-ae19-f530dff13712 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:44.370354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db85e41-8ac6-475a-ae19-f530dff13712 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:44.486421 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c2a34-e325-4970-973e-6491e2479289 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:44.488715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-240c2a34-e325-4970-973e-6491e2479289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:44.490018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 897/1813] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:44 2022] POST /identity/v3/projects => generated 367 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:44.493296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db85e41-8ac6-475a-ae19-f530dff13712 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:44.513632 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240c2a34-e325-4970-973e-6491e2479289 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:44.586564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db85e41-8ac6-475a-ae19-f530dff13712 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:44.591747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c750255-4f16-4fa8-9a18-6a02422a47d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:44.592393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c750255-4f16-4fa8-9a18-6a02422a47d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:44.592660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c750255-4f16-4fa8-9a18-6a02422a47d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:44.592931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c750255-4f16-4fa8-9a18-6a02422a47d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:44.593599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c750255-4f16-4fa8-9a18-6a02422a47d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:44.595624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c750255-4f16-4fa8-9a18-6a02422a47d3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:44.605400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0083d61-2a4a-45be-81c9-ea28204704eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:44.606065 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0083d61-2a4a-45be-81c9-ea28204704eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:44.606432 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0083d61-2a4a-45be-81c9-ea28204704eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:44.606822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0083d61-2a4a-45be-81c9-ea28204704eb admin admin] PATH_INFO: `/v3/projects/95b86810df6048b7b95b05363197826f` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:44.608321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0083d61-2a4a-45be-81c9-ea28204704eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=95b86810df6048b7b95b05363197826f)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:44.610812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0083d61-2a4a-45be-81c9-ea28204704eb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:44.726778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 898/1814] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:44 2022] POST /identity/v3/users => generated 284 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:03:44.731031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c750255-4f16-4fa8-9a18-6a02422a47d3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:44.742305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c750255-4f16-4fa8-9a18-6a02422a47d3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:44.752615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0083d61-2a4a-45be-81c9-ea28204704eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95b86810df6048b7b95b05363197826f', 'request_id': 'req-e0083d61-2a4a-45be-81c9-ea28204704eb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:44.761991 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0083d61-2a4a-45be-81c9-ea28204704eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95b86810df6048b7b95b05363197826f', 'request_id': 'req-e0083d61-2a4a-45be-81c9-ea28204704eb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:44.804963 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c412563-c88c-4065-9fab-b60e8909f8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:44.805581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c412563-c88c-4065-9fab-b60e8909f8f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:44.805897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c412563-c88c-4065-9fab-b60e8909f8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:44.806226 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c412563-c88c-4065-9fab-b60e8909f8f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:44.806872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c412563-c88c-4065-9fab-b60e8909f8f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:44.808553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c412563-c88c-4065-9fab-b60e8909f8f5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:44.818602 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 899/1815] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:44.824860 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c412563-c88c-4065-9fab-b60e8909f8f5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:44.835526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c412563-c88c-4065-9fab-b60e8909f8f5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:44.840358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520dd45c-0428-47bb-b18c-64b2af9d06b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:44.840985 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dd45c-0428-47bb-b18c-64b2af9d06b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:44.841314 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dd45c-0428-47bb-b18c-64b2af9d06b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:44.841982 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520dd45c-0428-47bb-b18c-64b2af9d06b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:44.842214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520dd45c-0428-47bb-b18c-64b2af9d06b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:44.843843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520dd45c-0428-47bb-b18c-64b2af9d06b6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:44.854100 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 900/1816] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:03:44.858252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520dd45c-0428-47bb-b18c-64b2af9d06b6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:44.869164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520dd45c-0428-47bb-b18c-64b2af9d06b6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:44.875467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e16d26-7c6b-44a9-ae4f-4ea270c2bef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:44.876155 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e16d26-7c6b-44a9-ae4f-4ea270c2bef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:44.878461 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e16d26-7c6b-44a9-ae4f-4ea270c2bef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:44.878461 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e16d26-7c6b-44a9-ae4f-4ea270c2bef4 admin admin] PATH_INFO: `/v3/projects/13607a27c6334c7b8f067b6c99115d45/users/73fcca66226846339da33af9200857a0/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:44.893539 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e16d26-7c6b-44a9-ae4f-4ea270c2bef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13607a27c6334c7b8f067b6c99115d45, user_id=73fcca66226846339da33af9200857a0, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:44.896274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e16d26-7c6b-44a9-ae4f-4ea270c2bef4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:44.908484 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0083d61-2a4a-45be-81c9-ea28204704eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95b86810df6048b7b95b05363197826f', 'request_id': 'req-e0083d61-2a4a-45be-81c9-ea28204704eb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:44.919835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0083d61-2a4a-45be-81c9-ea28204704eb admin admin] The token cache is being invalidate because project 95b86810df6048b7b95b05363197826f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:03:44.920129 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0083d61-2a4a-45be-81c9-ea28204704eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:44.920800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0083d61-2a4a-45be-81c9-ea28204704eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:44.922337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 917/1817] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:03:44 2022] DELETE /identity/v3/projects/95b86810df6048b7b95b05363197826f => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:44.927407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84642a2-8fab-4946-9715-415e32c91205 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:44.927740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84642a2-8fab-4946-9715-415e32c91205 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:44.928010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84642a2-8fab-4946-9715-415e32c91205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:44.948493 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a84642a2-8fab-4946-9715-415e32c91205 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:44.949709 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a84642a2-8fab-4946-9715-415e32c91205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:44.958017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 918/1818] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:44.969539 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990a1e0-a3de-401a-9c01-969b7ed51309 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:44.969826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990a1e0-a3de-401a-9c01-969b7ed51309 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:44.970128 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990a1e0-a3de-401a-9c01-969b7ed51309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:44.992207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6990a1e0-a3de-401a-9c01-969b7ed51309 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:44.993455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6990a1e0-a3de-401a-9c01-969b7ed51309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:45.002186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 919/1819] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:45.007052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990a1e0-a3de-401a-9c01-969b7ed51309 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:45.021990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990a1e0-a3de-401a-9c01-969b7ed51309 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:45.027163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 901/1820] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:44 2022] PUT /identity/v3/projects/13607a27c6334c7b8f067b6c99115d45/users/73fcca66226846339da33af9200857a0/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:45.039617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5d062-72db-4237-ac98-af4d84b28a1f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:45.040024 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5d062-72db-4237-ac98-af4d84b28a1f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:45.040346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb5d062-72db-4237-ac98-af4d84b28a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:45.043038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:45.070250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8eb5d062-72db-4237-ac98-af4d84b28a1f None None] MFA Rules not processed for user `73fcca66226846339da33af9200857a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:45.071488 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8eb5d062-72db-4237-ac98-af4d84b28a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:45.085592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b34421-9e7e-425b-af09-f0da2be4a36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:45.086210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b34421-9e7e-425b-af09-f0da2be4a36d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:45.086494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b34421-9e7e-425b-af09-f0da2be4a36d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:45.086770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b34421-9e7e-425b-af09-f0da2be4a36d admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:45.087422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b34421-9e7e-425b-af09-f0da2be4a36d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:45.087792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b34421-9e7e-425b-af09-f0da2be4a36d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:45.089502 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b34421-9e7e-425b-af09-f0da2be4a36d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:45.106899 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 920/1821] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:03:45.110718 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b34421-9e7e-425b-af09-f0da2be4a36d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:45.122644 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b34421-9e7e-425b-af09-f0da2be4a36d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:45.128317 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bca35-912b-4c3a-847f-d2b4016c916b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:45.128930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bca35-912b-4c3a-847f-d2b4016c916b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:45.129342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bca35-912b-4c3a-847f-d2b4016c916b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:45.129621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9bca35-912b-4c3a-847f-d2b4016c916b admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:45.130319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bca35-912b-4c3a-847f-d2b4016c916b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:45.132436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9bca35-912b-4c3a-847f-d2b4016c916b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:45.146125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 902/1822] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:45.155796 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5d062-72db-4237-ac98-af4d84b28a1f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:45.157206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8eb5d062-72db-4237-ac98-af4d84b28a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:45.171902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb5d062-72db-4237-ac98-af4d84b28a1f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:45.176815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d4a13-67e5-4d46-b9d9-acdc1b71c66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:45.177440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d4a13-67e5-4d46-b9d9-acdc1b71c66a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:45.177725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d4a13-67e5-4d46-b9d9-acdc1b71c66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:45.177999 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d4a13-67e5-4d46-b9d9-acdc1b71c66a admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:45.179200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d4a13-67e5-4d46-b9d9-acdc1b71c66a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:45.181137 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d4a13-67e5-4d46-b9d9-acdc1b71c66a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:45.253727 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 921/1823] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:45 2022] POST /identity/v3/projects => generated 409 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:03:45.261548 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bca35-912b-4c3a-847f-d2b4016c916b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:45.294468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 903/1824] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:45 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:03:45.300772 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d4a13-67e5-4d46-b9d9-acdc1b71c66a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:45.342227 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9bca35-912b-4c3a-847f-d2b4016c916b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:45.393799 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d4a13-67e5-4d46-b9d9-acdc1b71c66a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:45.399970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b842f4-bcd7-4508-b6cf-559ac1f8a3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:45.400590 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b842f4-bcd7-4508-b6cf-559ac1f8a3ef admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:45.400903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b842f4-bcd7-4508-b6cf-559ac1f8a3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:45.401319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b842f4-bcd7-4508-b6cf-559ac1f8a3ef admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:45.401965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b842f4-bcd7-4508-b6cf-559ac1f8a3ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:45.403921 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b842f4-bcd7-4508-b6cf-559ac1f8a3ef admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:45.433820 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c48005-01c1-4e48-8853-1e58c6bb57f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:45.434451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c48005-01c1-4e48-8853-1e58c6bb57f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:45.434734 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c48005-01c1-4e48-8853-1e58c6bb57f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:45.435007 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c48005-01c1-4e48-8853-1e58c6bb57f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:45.435734 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c48005-01c1-4e48-8853-1e58c6bb57f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:45.437802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c48005-01c1-4e48-8853-1e58c6bb57f4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:45.539283 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 904/1825] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:45 2022] POST /identity/v3/users => generated 289 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:45.540956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b842f4-bcd7-4508-b6cf-559ac1f8a3ef admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:45.556163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b842f4-bcd7-4508-b6cf-559ac1f8a3ef admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:45.560976 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d5711-fecd-4f86-801d-72e7d74eb9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:45.561697 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d5711-fecd-4f86-801d-72e7d74eb9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:45.562013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d5711-fecd-4f86-801d-72e7d74eb9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:45.562290 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597d5711-fecd-4f86-801d-72e7d74eb9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:45.562945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d5711-fecd-4f86-801d-72e7d74eb9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:45.564892 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597d5711-fecd-4f86-801d-72e7d74eb9a4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:45.582360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 905/1826] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:45.586603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 922/1827] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:45 2022] POST /identity/v3/users => generated 305 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:03:45.587671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d5711-fecd-4f86-801d-72e7d74eb9a4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:45.592629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c48005-01c1-4e48-8853-1e58c6bb57f4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:45.604898 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c48005-01c1-4e48-8853-1e58c6bb57f4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:45.610110 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597d5711-fecd-4f86-801d-72e7d74eb9a4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:45.616586 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac973269-f183-4ec6-b53d-fc950e66112a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:45.617931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd4b19-08b2-4306-b5b7-222b0636b1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:45.618892 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd4b19-08b2-4306-b5b7-222b0636b1ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:45.619171 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd4b19-08b2-4306-b5b7-222b0636b1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:45.619276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac973269-f183-4ec6-b53d-fc950e66112a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:45.619481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd4b19-08b2-4306-b5b7-222b0636b1ec admin admin] PATH_INFO: `/v3/projects/0cce71b499c84857b5993fd0de8c4c79/users/272e5409da6a47f692fc4befd98c4c4e/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:45.619706 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac973269-f183-4ec6-b53d-fc950e66112a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:45.620003 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac973269-f183-4ec6-b53d-fc950e66112a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:45.620957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac973269-f183-4ec6-b53d-fc950e66112a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:45.623238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac973269-f183-4ec6-b53d-fc950e66112a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:45.644602 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 923/1828] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:03:45.645905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac973269-f183-4ec6-b53d-fc950e66112a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:45.651575 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd4b19-08b2-4306-b5b7-222b0636b1ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cce71b499c84857b5993fd0de8c4c79, user_id=272e5409da6a47f692fc4befd98c4c4e, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:45.654922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd4b19-08b2-4306-b5b7-222b0636b1ec admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:45.660418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac973269-f183-4ec6-b53d-fc950e66112a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:45.669237 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e23ba-d884-49a5-8e13-55fcda61860e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:45.676419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e23ba-d884-49a5-8e13-55fcda61860e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:45.676419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e23ba-d884-49a5-8e13-55fcda61860e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:45.676419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e23ba-d884-49a5-8e13-55fcda61860e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:45.676419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e23ba-d884-49a5-8e13-55fcda61860e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:45.676419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80e23ba-d884-49a5-8e13-55fcda61860e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:45.685786 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 924/1829] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:45.690514 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e23ba-d884-49a5-8e13-55fcda61860e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:45.749348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e23ba-d884-49a5-8e13-55fcda61860e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:45.749348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76589c3c-2052-4ea0-93be-6516304de9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:45.749348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76589c3c-2052-4ea0-93be-6516304de9b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:45.749348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76589c3c-2052-4ea0-93be-6516304de9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:45.749673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76589c3c-2052-4ea0-93be-6516304de9b0 admin admin] PATH_INFO: `/v3/projects/4dec59e1057c4a6d80118b3116ba5276/users/0bfe79a180454ed4984fb8aa20fef175/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:45.749673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76589c3c-2052-4ea0-93be-6516304de9b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dec59e1057c4a6d80118b3116ba5276, user_id=0bfe79a180454ed4984fb8aa20fef175, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:45.749673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76589c3c-2052-4ea0-93be-6516304de9b0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:46.128488 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 906/1830] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:45 2022] PUT /identity/v3/projects/0cce71b499c84857b5993fd0de8c4c79/users/272e5409da6a47f692fc4befd98c4c4e/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:46.132423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd4b19-08b2-4306-b5b7-222b0636b1ec admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:46.242305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd4b19-08b2-4306-b5b7-222b0636b1ec admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:46.247773 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c5e52-0d58-4fad-8f29-02f891f43ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:46.248410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c5e52-0d58-4fad-8f29-02f891f43ead admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:46.248694 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c5e52-0d58-4fad-8f29-02f891f43ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:46.248969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c5e52-0d58-4fad-8f29-02f891f43ead admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:46.249625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c5e52-0d58-4fad-8f29-02f891f43ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:46.251323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c5e52-0d58-4fad-8f29-02f891f43ead admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:46.268370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 907/1831] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:03:46.274017 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c5e52-0d58-4fad-8f29-02f891f43ead admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:46.291177 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c5e52-0d58-4fad-8f29-02f891f43ead admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:46.296620 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528f7dd3-cdcc-4462-9f4f-67ebfa11d185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:46.297722 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f7dd3-cdcc-4462-9f4f-67ebfa11d185 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:46.298105 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f7dd3-cdcc-4462-9f4f-67ebfa11d185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:46.298487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528f7dd3-cdcc-4462-9f4f-67ebfa11d185 admin admin] PATH_INFO: `/v3/projects/0cce71b499c84857b5993fd0de8c4c79/users/272e5409da6a47f692fc4befd98c4c4e/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:46.302624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f7dd3-cdcc-4462-9f4f-67ebfa11d185 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cce71b499c84857b5993fd0de8c4c79, user_id=272e5409da6a47f692fc4befd98c4c4e, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:46.306791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528f7dd3-cdcc-4462-9f4f-67ebfa11d185 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:46.320070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 908/1832] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:46 2022] PUT /identity/v3/projects/0cce71b499c84857b5993fd0de8c4c79/users/272e5409da6a47f692fc4befd98c4c4e/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:46.339288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63d32d-821d-480e-b4d2-4f9b10b4aa6c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:46.339610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63d32d-821d-480e-b4d2-4f9b10b4aa6c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:46.339882 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c63d32d-821d-480e-b4d2-4f9b10b4aa6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:46.380164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 925/1833] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:45 2022] PUT /identity/v3/projects/4dec59e1057c4a6d80118b3116ba5276/users/0bfe79a180454ed4984fb8aa20fef175/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 681 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:46.384177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a89d0f-ecae-4b6e-9508-527b6326bb12 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:46.384669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a89d0f-ecae-4b6e-9508-527b6326bb12 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:46.385338 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a89d0f-ecae-4b6e-9508-527b6326bb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:46.417459 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7c63d32d-821d-480e-b4d2-4f9b10b4aa6c None None] MFA Rules not processed for user `272e5409da6a47f692fc4befd98c4c4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:46.418713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7c63d32d-821d-480e-b4d2-4f9b10b4aa6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:46.473228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b9a89d0f-ecae-4b6e-9508-527b6326bb12 None None] MFA Rules not processed for user `0bfe79a180454ed4984fb8aa20fef175`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:46.492363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b9a89d0f-ecae-4b6e-9508-527b6326bb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:46.582204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 909/1834] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:46 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:46.668479 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 926/1835] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:46 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:46.672998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63d32d-821d-480e-b4d2-4f9b10b4aa6c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:46.808659 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c63d32d-821d-480e-b4d2-4f9b10b4aa6c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:46.837700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5310206-11f1-459c-971a-af689712a7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:46.838466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5310206-11f1-459c-971a-af689712a7e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:46.838818 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5310206-11f1-459c-971a-af689712a7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:46.839169 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5310206-11f1-459c-971a-af689712a7e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:46.840559 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5310206-11f1-459c-971a-af689712a7e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:46.847964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5310206-11f1-459c-971a-af689712a7e1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:48.408113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 910/1836] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:46 2022] POST /identity/v3/projects => generated 409 bytes in 1733 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:03:48.410293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a89d0f-ecae-4b6e-9508-527b6326bb12 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:48.430419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a89d0f-ecae-4b6e-9508-527b6326bb12 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:48.439530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5310206-11f1-459c-971a-af689712a7e1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:48.442042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b5310206-11f1-459c-971a-af689712a7e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:48.517570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f681e8-1765-422b-99c7-f770504165ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:48.518714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f681e8-1765-422b-99c7-f770504165ff admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:48.519594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f681e8-1765-422b-99c7-f770504165ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:48.519905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f681e8-1765-422b-99c7-f770504165ff admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:48.520601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f681e8-1765-422b-99c7-f770504165ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:48.530844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f681e8-1765-422b-99c7-f770504165ff admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:48.572754 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5310206-11f1-459c-971a-af689712a7e1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:48.576715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e325a0c-bed8-4c17-a663-56ab9c818b8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:48.577337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e325a0c-bed8-4c17-a663-56ab9c818b8e service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:48.577615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e325a0c-bed8-4c17-a663-56ab9c818b8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:48.578076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e325a0c-bed8-4c17-a663-56ab9c818b8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:48.674444 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e325a0c-bed8-4c17-a663-56ab9c818b8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:48.675922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e325a0c-bed8-4c17-a663-56ab9c818b8e service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:48.693483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 911/1837] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:48.734620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 927/1838] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:48 2022] POST /identity/v3/users => generated 311 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:48.738955 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e325a0c-bed8-4c17-a663-56ab9c818b8e service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:48.759198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e325a0c-bed8-4c17-a663-56ab9c818b8e service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:48.764456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204075c-5b89-4936-b0a5-6582e740528f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:48.765269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204075c-5b89-4936-b0a5-6582e740528f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:48.765611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204075c-5b89-4936-b0a5-6582e740528f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:48.765917 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9204075c-5b89-4936-b0a5-6582e740528f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:48.766798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204075c-5b89-4936-b0a5-6582e740528f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:48.768491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9204075c-5b89-4936-b0a5-6582e740528f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:48.779834 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 912/1839] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:48.784209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f681e8-1765-422b-99c7-f770504165ff admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:48.797131 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f681e8-1765-422b-99c7-f770504165ff admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:48.802077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6e363-85aa-4282-8851-32d2b5db478a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:48.802812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6e363-85aa-4282-8851-32d2b5db478a admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:48.803777 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6e363-85aa-4282-8851-32d2b5db478a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:48.804165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c6e363-85aa-4282-8851-32d2b5db478a admin admin] PATH_INFO: `/v3/projects/5b81eed50e0d4b0bb05a5b00c6be1818/users/de99b26d12bb4bc9b8b1191663fe3220/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:48.820912 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6e363-85aa-4282-8851-32d2b5db478a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b81eed50e0d4b0bb05a5b00c6be1818, user_id=de99b26d12bb4bc9b8b1191663fe3220, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:48.826168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c6e363-85aa-4282-8851-32d2b5db478a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:48.954151 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:48.967005 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 928/1840] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:48 2022] PUT /identity/v3/projects/5b81eed50e0d4b0bb05a5b00c6be1818/users/de99b26d12bb4bc9b8b1191663fe3220/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:48.971103 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204075c-5b89-4936-b0a5-6582e740528f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:49.004523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9204075c-5b89-4936-b0a5-6582e740528f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:49.116570 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412d18e-aa72-421d-8937-ce2e1df7d652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:49.118256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412d18e-aa72-421d-8937-ce2e1df7d652 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:49.118256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412d18e-aa72-421d-8937-ce2e1df7d652 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:49.119017 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412d18e-aa72-421d-8937-ce2e1df7d652 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:49.119481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412d18e-aa72-421d-8937-ce2e1df7d652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:49.121685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412d18e-aa72-421d-8937-ce2e1df7d652 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:49.146165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 913/1841] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:49.150468 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6e363-85aa-4282-8851-32d2b5db478a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:49.169894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c6e363-85aa-4282-8851-32d2b5db478a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:49.180584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3fe0f-9f9a-464e-ba3f-e9f203b9a610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:49.181413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3fe0f-9f9a-464e-ba3f-e9f203b9a610 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:49.181694 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3fe0f-9f9a-464e-ba3f-e9f203b9a610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:49.181970 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3fe0f-9f9a-464e-ba3f-e9f203b9a610 admin admin] PATH_INFO: `/v3/projects/5b81eed50e0d4b0bb05a5b00c6be1818/users/de99b26d12bb4bc9b8b1191663fe3220/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:49.183998 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3fe0f-9f9a-464e-ba3f-e9f203b9a610 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b81eed50e0d4b0bb05a5b00c6be1818, user_id=de99b26d12bb4bc9b8b1191663fe3220, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:49.187940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3fe0f-9f9a-464e-ba3f-e9f203b9a610 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:49.217036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 929/1842] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:49 2022] PUT /identity/v3/projects/5b81eed50e0d4b0bb05a5b00c6be1818/users/de99b26d12bb4bc9b8b1191663fe3220/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:49.218528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e906d0-635d-498a-8fda-282cc1f438c8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:49.218818 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e906d0-635d-498a-8fda-282cc1f438c8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:49.219096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e906d0-635d-498a-8fda-282cc1f438c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:49.276268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-92e906d0-635d-498a-8fda-282cc1f438c8 None None] MFA Rules not processed for user `de99b26d12bb4bc9b8b1191663fe3220`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:49.293861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-92e906d0-635d-498a-8fda-282cc1f438c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:49.360009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3fe0f-9f9a-464e-ba3f-e9f203b9a610 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:49.482257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 914/1843] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:49 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:49.529957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3fe0f-9f9a-464e-ba3f-e9f203b9a610 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:49.542257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c94cb4-c578-4807-b536-4182d5255f5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:49.542892 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c94cb4-c578-4807-b536-4182d5255f5c service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:49.543176 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c94cb4-c578-4807-b536-4182d5255f5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:49.543487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c94cb4-c578-4807-b536-4182d5255f5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:49.633784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0f9d1-83f5-4adc-b53b-16fef53ace0b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:49.634071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0f9d1-83f5-4adc-b53b-16fef53ace0b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:49.634351 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e0f9d1-83f5-4adc-b53b-16fef53ace0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:49.674602 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-44e0f9d1-83f5-4adc-b53b-16fef53ace0b None None] MFA Rules not processed for user `ef7e0b772a284e7da4fae7d73f34d3de`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:49.678738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-44e0f9d1-83f5-4adc-b53b-16fef53ace0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:49.693271 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c94cb4-c578-4807-b536-4182d5255f5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:49.694087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c94cb4-c578-4807-b536-4182d5255f5c service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:49.726158 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 930/1844] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:49 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:49.727603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c94cb4-c578-4807-b536-4182d5255f5c service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:49.729594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b7c94cb4-c578-4807-b536-4182d5255f5c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:49.746132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c94cb4-c578-4807-b536-4182d5255f5c service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:49.750754 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd95a5-38d8-4447-a4ae-4ffd6ba2efcc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:49.751416 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd95a5-38d8-4447-a4ae-4ffd6ba2efcc service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:49.751772 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd95a5-38d8-4447-a4ae-4ffd6ba2efcc service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:49.752014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd95a5-38d8-4447-a4ae-4ffd6ba2efcc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:49.769550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd95a5-38d8-4447-a4ae-4ffd6ba2efcc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:49.774223 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd95a5-38d8-4447-a4ae-4ffd6ba2efcc service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:49.799200 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 931/1845] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:49 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:49.837428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 915/1846] 10.222.0.88 () {58 vars in 1015 bytes} [Mon Sep 26 18:03:49 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:50.014844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd95a5-38d8-4447-a4ae-4ffd6ba2efcc service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:50.016888 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-65cd95a5-38d8-4447-a4ae-4ffd6ba2efcc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:50.042601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd95a5-38d8-4447-a4ae-4ffd6ba2efcc service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:50.047617 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262d7956-ded2-4882-a3fa-20b9d3213972 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:50.048696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d7956-ded2-4882-a3fa-20b9d3213972 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:50.049352 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d7956-ded2-4882-a3fa-20b9d3213972 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:50.050319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262d7956-ded2-4882-a3fa-20b9d3213972 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:50.060198 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d7956-ded2-4882-a3fa-20b9d3213972 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:50.062471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262d7956-ded2-4882-a3fa-20b9d3213972 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:50.074127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 932/1847] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:50.824756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0f9d1-83f5-4adc-b53b-16fef53ace0b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:50.843395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e0f9d1-83f5-4adc-b53b-16fef53ace0b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:50.847879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0678810-89d7-41d1-a65f-f87bb4283290 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:50.848950 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0678810-89d7-41d1-a65f-f87bb4283290 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:50.849513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0678810-89d7-41d1-a65f-f87bb4283290 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:50.850193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0678810-89d7-41d1-a65f-f87bb4283290 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:50.946483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0678810-89d7-41d1-a65f-f87bb4283290 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:50.949584 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0678810-89d7-41d1-a65f-f87bb4283290 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:50.961972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 916/1848] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:50 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:51.595094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:51.797682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368ca57-167f-432a-a902-895c8e4b34b8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:51.797968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368ca57-167f-432a-a902-895c8e4b34b8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:51.798245 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b368ca57-167f-432a-a902-895c8e4b34b8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:51.800983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 933/1849] 10.222.0.67 () {54 vars in 921 bytes} [Mon Sep 26 18:03:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:03:51.807626 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7d3a2-8c16-408b-87c3-f8c9b6553679 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:51.808228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7d3a2-8c16-408b-87c3-f8c9b6553679 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:51.808512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7d3a2-8c16-408b-87c3-f8c9b6553679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:51.866515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d5c7d3a2-8c16-408b-87c3-f8c9b6553679 None None] MFA Rules not processed for user `b9ed29a3f6be4d64bb3751f9e95eb56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:51.888809 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d5c7d3a2-8c16-408b-87c3-f8c9b6553679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:52.096039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 917/1850] 10.222.0.67 () {58 vars in 1018 bytes} [Mon Sep 26 18:03:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:52.125653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368ca57-167f-432a-a902-895c8e4b34b8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:52.130130 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b368ca57-167f-432a-a902-895c8e4b34b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:52.175067 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b368ca57-167f-432a-a902-895c8e4b34b8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:52.178713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26f60b5-00ac-49ba-9631-4e8b94b43565 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:52.180951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f60b5-00ac-49ba-9631-4e8b94b43565 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:52.181563 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f60b5-00ac-49ba-9631-4e8b94b43565 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:52.182464 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26f60b5-00ac-49ba-9631-4e8b94b43565 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:52.240176 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26f60b5-00ac-49ba-9631-4e8b94b43565 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:52.246382 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26f60b5-00ac-49ba-9631-4e8b94b43565 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:52.263636 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 934/1851] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:52.759110 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7d3a2-8c16-408b-87c3-f8c9b6553679 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:52.761118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d5c7d3a2-8c16-408b-87c3-f8c9b6553679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:52.994686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7d3a2-8c16-408b-87c3-f8c9b6553679 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:53.005691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd6cf9-461e-458a-b46d-3e3a0d94d0b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:53.007221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd6cf9-461e-458a-b46d-3e3a0d94d0b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:53.007221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd6cf9-461e-458a-b46d-3e3a0d94d0b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:53.007221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd6cf9-461e-458a-b46d-3e3a0d94d0b2 admin admin] PATH_INFO: `/v3/users/152105c81a1a4c989d41f9a0fbc6d68c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:53.009610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd6cf9-461e-458a-b46d-3e3a0d94d0b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=152105c81a1a4c989d41f9a0fbc6d68c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:53.014213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd6cf9-461e-458a-b46d-3e3a0d94d0b2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:53.159359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036ce5f-3515-4ea3-bbaa-167ddf8fcec0 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:53.159887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036ce5f-3515-4ea3-bbaa-167ddf8fcec0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:53.167756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036ce5f-3515-4ea3-bbaa-167ddf8fcec0 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:53.167756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 935/1852] 10.222.0.67 () {54 vars in 921 bytes} [Mon Sep 26 18:03:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 18:03:53.188784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88240a-929f-4cde-8a3f-23d97419de1a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:53.189082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88240a-929f-4cde-8a3f-23d97419de1a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:53.195937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88240a-929f-4cde-8a3f-23d97419de1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:53.249762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2a88240a-929f-4cde-8a3f-23d97419de1a None None] MFA Rules not processed for user `b9ed29a3f6be4d64bb3751f9e95eb56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:53.251599 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2a88240a-929f-4cde-8a3f-23d97419de1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:53.270390 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 936/1853] 10.222.0.67 () {58 vars in 1018 bytes} [Mon Sep 26 18:03:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:53.377267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-adcd6cf9-461e-458a-b46d-3e3a0d94d0b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '152105c81a1a4c989d41f9a0fbc6d68c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:53.515486 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-adcd6cf9-461e-458a-b46d-3e3a0d94d0b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '152105c81a1a4c989d41f9a0fbc6d68c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:53.516427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-adcd6cf9-461e-458a-b46d-3e3a0d94d0b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '152105c81a1a4c989d41f9a0fbc6d68c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:53.518066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-adcd6cf9-461e-458a-b46d-3e3a0d94d0b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '152105c81a1a4c989d41f9a0fbc6d68c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:53.563863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-adcd6cf9-461e-458a-b46d-3e3a0d94d0b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '152105c81a1a4c989d41f9a0fbc6d68c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:53.595441 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88240a-929f-4cde-8a3f-23d97419de1a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:53.600222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 918/1854] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:03:52 2022] DELETE /identity/v3/users/152105c81a1a4c989d41f9a0fbc6d68c => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:53.603261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52deeca0-3cb7-4f11-af06-06f529987f37 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:53.603666 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52deeca0-3cb7-4f11-af06-06f529987f37 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:53.603924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52deeca0-3cb7-4f11-af06-06f529987f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:53.609429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2a88240a-929f-4cde-8a3f-23d97419de1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:53.648280 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88240a-929f-4cde-8a3f-23d97419de1a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:53.663975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-52deeca0-3cb7-4f11-af06-06f529987f37 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:53.669680 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-52deeca0-3cb7-4f11-af06-06f529987f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:53.828489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65a818-3073-405c-be91-e8aff98a007c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:53.829615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65a818-3073-405c-be91-e8aff98a007c service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:53.830278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65a818-3073-405c-be91-e8aff98a007c service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:53.831191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65a818-3073-405c-be91-e8aff98a007c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:53.839563 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2c65a818-3073-405c-be91-e8aff98a007c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:53.880840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 919/1855] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:03:53.913781 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65a818-3073-405c-be91-e8aff98a007c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:53.916827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52deeca0-3cb7-4f11-af06-06f529987f37 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:53.919081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c65a818-3073-405c-be91-e8aff98a007c service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:53.933644 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-52deeca0-3cb7-4f11-af06-06f529987f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:53.976953 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 937/1856] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:54.231070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52deeca0-3cb7-4f11-af06-06f529987f37 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:54.236448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b7c5a-6c3a-4cf6-b8ce-70fe6acf0fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:54.237244 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b7c5a-6c3a-4cf6-b8ce-70fe6acf0fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:54.237690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b7c5a-6c3a-4cf6-b8ce-70fe6acf0fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:54.238035 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b7c5a-6c3a-4cf6-b8ce-70fe6acf0fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:54.286710 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b7c5a-6c3a-4cf6-b8ce-70fe6acf0fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:54.292957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b7c5a-6c3a-4cf6-b8ce-70fe6acf0fe3 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:54.345994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 920/1857] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:03:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:03:55.117287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65a818-3073-405c-be91-e8aff98a007c service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:55.121194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2c65a818-3073-405c-be91-e8aff98a007c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:55.156139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65a818-3073-405c-be91-e8aff98a007c service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:55.161778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c8910-9142-41ec-896d-474547e1d622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:55.163424 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c8910-9142-41ec-896d-474547e1d622 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:55.164512 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c8910-9142-41ec-896d-474547e1d622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:55.165239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c8910-9142-41ec-896d-474547e1d622 admin admin] PATH_INFO: `/v3/projects/1ce539f13fe8493fa0fe48620d9e1dea` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:55.171040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c8910-9142-41ec-896d-474547e1d622 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ce539f13fe8493fa0fe48620d9e1dea)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:55.173642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c8910-9142-41ec-896d-474547e1d622 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:55.311529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-143c8910-9142-41ec-896d-474547e1d622 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ce539f13fe8493fa0fe48620d9e1dea', 'request_id': 'req-143c8910-9142-41ec-896d-474547e1d622'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:55.320333 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-143c8910-9142-41ec-896d-474547e1d622 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ce539f13fe8493fa0fe48620d9e1dea', 'request_id': 'req-143c8910-9142-41ec-896d-474547e1d622'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:55.444844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-143c8910-9142-41ec-896d-474547e1d622 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ce539f13fe8493fa0fe48620d9e1dea', 'request_id': 'req-143c8910-9142-41ec-896d-474547e1d622'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:55.460190 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-143c8910-9142-41ec-896d-474547e1d622 admin admin] The token cache is being invalidate because project 1ce539f13fe8493fa0fe48620d9e1dea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:03:55.460492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-143c8910-9142-41ec-896d-474547e1d622 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:55.461205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-143c8910-9142-41ec-896d-474547e1d622 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:55.462806 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 938/1858] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:03:55 2022] DELETE /identity/v3/projects/1ce539f13fe8493fa0fe48620d9e1dea => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:55.467447 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b7c5a-6c3a-4cf6-b8ce-70fe6acf0fe3 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:55.480408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b7c5a-6c3a-4cf6-b8ce-70fe6acf0fe3 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:55.519497 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:55.569636 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:55.595835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df729e2e-a2c4-48bb-b27f-82b99d874700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:55.596506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df729e2e-a2c4-48bb-b27f-82b99d874700 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:55.596792 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df729e2e-a2c4-48bb-b27f-82b99d874700 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:55.597072 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df729e2e-a2c4-48bb-b27f-82b99d874700 admin admin] PATH_INFO: `/v3/users/574859147d14417cb5f9fddfbbace502` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:55.601608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df729e2e-a2c4-48bb-b27f-82b99d874700 admin admin] RBAC: Authorizing `identity:delete_user(user_id=574859147d14417cb5f9fddfbbace502)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:55.604587 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df729e2e-a2c4-48bb-b27f-82b99d874700 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:55.918393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-df729e2e-a2c4-48bb-b27f-82b99d874700 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '574859147d14417cb5f9fddfbbace502'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:56.133143 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-df729e2e-a2c4-48bb-b27f-82b99d874700 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '574859147d14417cb5f9fddfbbace502'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:56.134437 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-df729e2e-a2c4-48bb-b27f-82b99d874700 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '574859147d14417cb5f9fddfbbace502'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:56.135799 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-df729e2e-a2c4-48bb-b27f-82b99d874700 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '574859147d14417cb5f9fddfbbace502'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:56.156637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-df729e2e-a2c4-48bb-b27f-82b99d874700 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '574859147d14417cb5f9fddfbbace502'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:56.186635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 921/1859] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:03:55 2022] DELETE /identity/v3/users/574859147d14417cb5f9fddfbbace502 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:56.992634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c8910-9142-41ec-896d-474547e1d622 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:57.001433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-143c8910-9142-41ec-896d-474547e1d622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:57.009965 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c8910-9142-41ec-896d-474547e1d622 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:57.127164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fb1c94-cb3c-4e5e-a621-03c014dbfc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:57.128258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb1c94-cb3c-4e5e-a621-03c014dbfc90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:57.133299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb1c94-cb3c-4e5e-a621-03c014dbfc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:57.133299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fb1c94-cb3c-4e5e-a621-03c014dbfc90 admin admin] PATH_INFO: `/v3/projects/8858ae8903bf45018b353a2f3948e963` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:57.133299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb1c94-cb3c-4e5e-a621-03c014dbfc90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8858ae8903bf45018b353a2f3948e963)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:57.136085 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fb1c94-cb3c-4e5e-a621-03c014dbfc90 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:57.332137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4fb1c94-cb3c-4e5e-a621-03c014dbfc90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8858ae8903bf45018b353a2f3948e963', 'request_id': 'req-e4fb1c94-cb3c-4e5e-a621-03c014dbfc90'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:57.341625 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4fb1c94-cb3c-4e5e-a621-03c014dbfc90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8858ae8903bf45018b353a2f3948e963', 'request_id': 'req-e4fb1c94-cb3c-4e5e-a621-03c014dbfc90'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:57.525122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4fb1c94-cb3c-4e5e-a621-03c014dbfc90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8858ae8903bf45018b353a2f3948e963', 'request_id': 'req-e4fb1c94-cb3c-4e5e-a621-03c014dbfc90'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:57.538778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4fb1c94-cb3c-4e5e-a621-03c014dbfc90 admin admin] The token cache is being invalidate because project 8858ae8903bf45018b353a2f3948e963 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:03:57.539081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4fb1c94-cb3c-4e5e-a621-03c014dbfc90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:57.539794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4fb1c94-cb3c-4e5e-a621-03c014dbfc90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:03:57.541697 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 939/1860] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:03:56 2022] DELETE /identity/v3/projects/8858ae8903bf45018b353a2f3948e963 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:03:57.554042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5923d1-0645-4ec9-bf8e-7fe3fb59650e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:57.554665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5923d1-0645-4ec9-bf8e-7fe3fb59650e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:57.554665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5923d1-0645-4ec9-bf8e-7fe3fb59650e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:57.577300 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7f5923d1-0645-4ec9-bf8e-7fe3fb59650e None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:57.578634 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7f5923d1-0645-4ec9-bf8e-7fe3fb59650e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:57.657916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 922/1861] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:57.667546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d113d57-34e0-4214-b066-f706778b7630 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:57.667992 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d113d57-34e0-4214-b066-f706778b7630 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:57.668344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d113d57-34e0-4214-b066-f706778b7630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:57.691811 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7d113d57-34e0-4214-b066-f706778b7630 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:57.699325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7d113d57-34e0-4214-b066-f706778b7630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:57.702675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 940/1862] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:57.707343 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5923d1-0645-4ec9-bf8e-7fe3fb59650e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:57.721137 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5923d1-0645-4ec9-bf8e-7fe3fb59650e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:57.732495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548ec9b-54cc-4241-89ce-30f57773f14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:57.733137 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548ec9b-54cc-4241-89ce-30f57773f14f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:57.733457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548ec9b-54cc-4241-89ce-30f57773f14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:57.733686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b548ec9b-54cc-4241-89ce-30f57773f14f admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:57.734763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548ec9b-54cc-4241-89ce-30f57773f14f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:03:57.735211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548ec9b-54cc-4241-89ce-30f57773f14f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:57.737590 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b548ec9b-54cc-4241-89ce-30f57773f14f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:57.754973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 923/1863] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:03:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:03:57.768130 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d113d57-34e0-4214-b066-f706778b7630 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:57.783987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d113d57-34e0-4214-b066-f706778b7630 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:57.798383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452cc1e-e42d-4b45-9c9f-74a2eb53bb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:57.799235 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452cc1e-e42d-4b45-9c9f-74a2eb53bb3c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:57.799490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452cc1e-e42d-4b45-9c9f-74a2eb53bb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:57.799891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452cc1e-e42d-4b45-9c9f-74a2eb53bb3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:57.800577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452cc1e-e42d-4b45-9c9f-74a2eb53bb3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:57.802827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452cc1e-e42d-4b45-9c9f-74a2eb53bb3c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:57.988117 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 941/1864] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:57 2022] POST /identity/v3/projects => generated 387 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:57.991267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548ec9b-54cc-4241-89ce-30f57773f14f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:58.023972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b548ec9b-54cc-4241-89ce-30f57773f14f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:58.204682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaab6a-6309-4789-9b80-7741ea8fef15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:58.204682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaab6a-6309-4789-9b80-7741ea8fef15 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:58.204682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaab6a-6309-4789-9b80-7741ea8fef15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:58.204682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abaab6a-6309-4789-9b80-7741ea8fef15 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:58.205456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abaab6a-6309-4789-9b80-7741ea8fef15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:58.205456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abaab6a-6309-4789-9b80-7741ea8fef15 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:58.331974 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 924/1865] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:57 2022] POST /identity/v3/users => generated 294 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:58.336659 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452cc1e-e42d-4b45-9c9f-74a2eb53bb3c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:58.350081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452cc1e-e42d-4b45-9c9f-74a2eb53bb3c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:58.360986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96721a03-7337-412d-bc3e-acb434d6e406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:58.361739 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96721a03-7337-412d-bc3e-acb434d6e406 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:58.362076 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96721a03-7337-412d-bc3e-acb434d6e406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:58.362499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96721a03-7337-412d-bc3e-acb434d6e406 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:58.363238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96721a03-7337-412d-bc3e-acb434d6e406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:58.365158 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96721a03-7337-412d-bc3e-acb434d6e406 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:58.380561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 942/1866] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:58.386893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaab6a-6309-4789-9b80-7741ea8fef15 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:58.400444 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abaab6a-6309-4789-9b80-7741ea8fef15 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:58.409938 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c33be66-94f6-4914-96b4-5922e4bc2496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:58.410444 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33be66-94f6-4914-96b4-5922e4bc2496 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:58.410792 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33be66-94f6-4914-96b4-5922e4bc2496 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:58.411142 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c33be66-94f6-4914-96b4-5922e4bc2496 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:58.412641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33be66-94f6-4914-96b4-5922e4bc2496 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:58.414786 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c33be66-94f6-4914-96b4-5922e4bc2496 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:58.429474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 925/1867] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:58.433585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96721a03-7337-412d-bc3e-acb434d6e406 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:58.444301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96721a03-7337-412d-bc3e-acb434d6e406 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:58.448893 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97907a-ce79-4c2c-a8ea-1aa46a977bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:58.449566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97907a-ce79-4c2c-a8ea-1aa46a977bd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:58.449847 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97907a-ce79-4c2c-a8ea-1aa46a977bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:58.450120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd97907a-ce79-4c2c-a8ea-1aa46a977bd4 admin admin] PATH_INFO: `/v3/projects/4b0f25b318054d85b6ab8d1bea54aaef/users/2f776652b501404fac49918c2d559d01/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:58.485910 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97907a-ce79-4c2c-a8ea-1aa46a977bd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b0f25b318054d85b6ab8d1bea54aaef, user_id=2f776652b501404fac49918c2d559d01, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:58.485910 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd97907a-ce79-4c2c-a8ea-1aa46a977bd4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:58.491927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:58.731792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 943/1868] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:58 2022] PUT /identity/v3/projects/4b0f25b318054d85b6ab8d1bea54aaef/users/2f776652b501404fac49918c2d559d01/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:58.742097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb0835-9a14-4ae1-a5a8-3a337c1d0b80 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:58.742926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb0835-9a14-4ae1-a5a8-3a337c1d0b80 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:58.742926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb0835-9a14-4ae1-a5a8-3a337c1d0b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:58.776118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6ffb0835-9a14-4ae1-a5a8-3a337c1d0b80 None None] MFA Rules not processed for user `2f776652b501404fac49918c2d559d01`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:03:58.782699 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6ffb0835-9a14-4ae1-a5a8-3a337c1d0b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:03:58.849768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 926/1869] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:03:58.854696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97907a-ce79-4c2c-a8ea-1aa46a977bd4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:58.935510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd97907a-ce79-4c2c-a8ea-1aa46a977bd4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:58.940246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d878d-4736-48a4-bb12-c404c8ae9c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:58.940858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d878d-4736-48a4-bb12-c404c8ae9c00 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:58.941132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d878d-4736-48a4-bb12-c404c8ae9c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:58.941397 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d878d-4736-48a4-bb12-c404c8ae9c00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:58.942039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d878d-4736-48a4-bb12-c404c8ae9c00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:58.944713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d878d-4736-48a4-bb12-c404c8ae9c00 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:59.162159 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 944/1870] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:03:58 2022] POST /identity/v3/projects => generated 389 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:59.204857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb0835-9a14-4ae1-a5a8-3a337c1d0b80 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:59.216796 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb0835-9a14-4ae1-a5a8-3a337c1d0b80 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:59.278881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56f5e4-aa75-41ed-99e2-6ad6e92b9d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:59.279428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56f5e4-aa75-41ed-99e2-6ad6e92b9d42 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:59.279741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56f5e4-aa75-41ed-99e2-6ad6e92b9d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:59.280020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56f5e4-aa75-41ed-99e2-6ad6e92b9d42 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:59.280668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56f5e4-aa75-41ed-99e2-6ad6e92b9d42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:59.282608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d56f5e4-aa75-41ed-99e2-6ad6e92b9d42 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:59.560858 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 927/1871] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:03:59 2022] POST /identity/v3/users => generated 301 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:03:59.564738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d878d-4736-48a4-bb12-c404c8ae9c00 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:59.575966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d878d-4736-48a4-bb12-c404c8ae9c00 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:59.580669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea492a20-7752-44b0-bc89-ba76c4a1972a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:59.587823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea492a20-7752-44b0-bc89-ba76c4a1972a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:59.587823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea492a20-7752-44b0-bc89-ba76c4a1972a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:59.587823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea492a20-7752-44b0-bc89-ba76c4a1972a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:59.587823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea492a20-7752-44b0-bc89-ba76c4a1972a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:59.587823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea492a20-7752-44b0-bc89-ba76c4a1972a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:59.596641 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 945/1872] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:59.600341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56f5e4-aa75-41ed-99e2-6ad6e92b9d42 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:59.610698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56f5e4-aa75-41ed-99e2-6ad6e92b9d42 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:59.615840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8bd1a-6d3c-4f67-8ca6-ec6ef914f8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:59.616616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8bd1a-6d3c-4f67-8ca6-ec6ef914f8e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:59.616967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8bd1a-6d3c-4f67-8ca6-ec6ef914f8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:59.617315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f8bd1a-6d3c-4f67-8ca6-ec6ef914f8e2 admin admin] PATH_INFO: `/v3/projects/02a75d7df1db4133b58088ac3ddac9ea/users/24df618b9abe40bc9f6302451f8b6fc7/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:59.634266 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8bd1a-6d3c-4f67-8ca6-ec6ef914f8e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02a75d7df1db4133b58088ac3ddac9ea, user_id=24df618b9abe40bc9f6302451f8b6fc7, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:59.637554 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f8bd1a-6d3c-4f67-8ca6-ec6ef914f8e2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:59.835201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 928/1873] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:59 2022] PUT /identity/v3/projects/02a75d7df1db4133b58088ac3ddac9ea/users/24df618b9abe40bc9f6302451f8b6fc7/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:59.839093 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea492a20-7752-44b0-bc89-ba76c4a1972a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:59.849428 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea492a20-7752-44b0-bc89-ba76c4a1972a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:59.893326 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:03:59.912822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66c51e-174e-4978-ab86-e5584ebc9c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:59.913549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66c51e-174e-4978-ab86-e5584ebc9c95 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:59.913832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66c51e-174e-4978-ab86-e5584ebc9c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:59.914109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66c51e-174e-4978-ab86-e5584ebc9c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:59.914747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66c51e-174e-4978-ab86-e5584ebc9c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:59.916380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66c51e-174e-4978-ab86-e5584ebc9c95 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:59.926178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 946/1874] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:03:59 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:03:59.930629 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8bd1a-6d3c-4f67-8ca6-ec6ef914f8e2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:03:59.941744 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f8bd1a-6d3c-4f67-8ca6-ec6ef914f8e2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:03:59.946903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb4380-1efb-4dbf-91d3-fe9e0ab1906c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:03:59.947686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb4380-1efb-4dbf-91d3-fe9e0ab1906c admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:59.948097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb4380-1efb-4dbf-91d3-fe9e0ab1906c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:59.948386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03eb4380-1efb-4dbf-91d3-fe9e0ab1906c admin admin] PATH_INFO: `/v3/projects/02a75d7df1db4133b58088ac3ddac9ea/users/24df618b9abe40bc9f6302451f8b6fc7/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:03:59.950340 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb4380-1efb-4dbf-91d3-fe9e0ab1906c admin admin] RBAC: Authorizing `identity:create_grant(project_id=02a75d7df1db4133b58088ac3ddac9ea, user_id=24df618b9abe40bc9f6302451f8b6fc7, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:03:59.954030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03eb4380-1efb-4dbf-91d3-fe9e0ab1906c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:03:59.965475 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 929/1875] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:03:59 2022] PUT /identity/v3/projects/02a75d7df1db4133b58088ac3ddac9ea/users/24df618b9abe40bc9f6302451f8b6fc7/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:03:59.977692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940b7b2-5bc2-4395-b655-83651e3b173d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:03:59.977986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940b7b2-5bc2-4395-b655-83651e3b173d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:03:59.978324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a940b7b2-5bc2-4395-b655-83651e3b173d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:00.007059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a940b7b2-5bc2-4395-b655-83651e3b173d None None] MFA Rules not processed for user `24df618b9abe40bc9f6302451f8b6fc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:00.008324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a940b7b2-5bc2-4395-b655-83651e3b173d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:00.079141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 947/1876] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:03:59 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:00.090791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb4380-1efb-4dbf-91d3-fe9e0ab1906c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:00.092471 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-03eb4380-1efb-4dbf-91d3-fe9e0ab1906c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:00.160806 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03eb4380-1efb-4dbf-91d3-fe9e0ab1906c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:00.165445 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2d52d-475e-441d-89cf-b10cc499ef41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:00.166078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2d52d-475e-441d-89cf-b10cc499ef41 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:00.166363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2d52d-475e-441d-89cf-b10cc499ef41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:00.167042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2d52d-475e-441d-89cf-b10cc499ef41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:00.177751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2d52d-475e-441d-89cf-b10cc499ef41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:00.179436 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2d52d-475e-441d-89cf-b10cc499ef41 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:00.189432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 930/1877] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:00 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:00.363570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940b7b2-5bc2-4395-b655-83651e3b173d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:00.431448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a940b7b2-5bc2-4395-b655-83651e3b173d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:00.436316 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34669943-590e-4b1d-bd84-53b44c360283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:00.436961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34669943-590e-4b1d-bd84-53b44c360283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:00.437239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34669943-590e-4b1d-bd84-53b44c360283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:00.437512 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34669943-590e-4b1d-bd84-53b44c360283 admin admin] PATH_INFO: `/v3/users/2f776652b501404fac49918c2d559d01` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:00.438650 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34669943-590e-4b1d-bd84-53b44c360283 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f776652b501404fac49918c2d559d01)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:00.440664 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34669943-590e-4b1d-bd84-53b44c360283 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:00.897969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34669943-590e-4b1d-bd84-53b44c360283 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f776652b501404fac49918c2d559d01'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:01.079378 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34669943-590e-4b1d-bd84-53b44c360283 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f776652b501404fac49918c2d559d01'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:01.080118 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34669943-590e-4b1d-bd84-53b44c360283 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f776652b501404fac49918c2d559d01'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:01.080712 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34669943-590e-4b1d-bd84-53b44c360283 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f776652b501404fac49918c2d559d01'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:01.089758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34669943-590e-4b1d-bd84-53b44c360283 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f776652b501404fac49918c2d559d01'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:01.109117 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 948/1878] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:00 2022] DELETE /identity/v3/users/2f776652b501404fac49918c2d559d01 => generated 0 bytes in 746 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:01.113796 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02179b06-087b-4acd-8d04-3102ad91b04b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:01.114844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02179b06-087b-4acd-8d04-3102ad91b04b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:01.115223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02179b06-087b-4acd-8d04-3102ad91b04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:01.138829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-02179b06-087b-4acd-8d04-3102ad91b04b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:01.140261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-02179b06-087b-4acd-8d04-3102ad91b04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:01.255008 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 931/1879] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:01.267062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34669943-590e-4b1d-bd84-53b44c360283 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:01.268926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-34669943-590e-4b1d-bd84-53b44c360283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:01.349896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34669943-590e-4b1d-bd84-53b44c360283 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:01.354529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07882e-c523-40d0-b576-c04dbff79cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:01.355592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07882e-c523-40d0-b576-c04dbff79cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:01.355997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07882e-c523-40d0-b576-c04dbff79cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:01.356378 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07882e-c523-40d0-b576-c04dbff79cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:01.371454 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07882e-c523-40d0-b576-c04dbff79cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:01.375393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07882e-c523-40d0-b576-c04dbff79cf6 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:01.387298 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 949/1880] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:01.943556 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:02.089417 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02179b06-087b-4acd-8d04-3102ad91b04b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:02.090966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-02179b06-087b-4acd-8d04-3102ad91b04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:02.104269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02179b06-087b-4acd-8d04-3102ad91b04b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:02.110459 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404217a-fdf9-4766-aad9-75627c17c5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:02.111234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404217a-fdf9-4766-aad9-75627c17c5fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:02.111519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404217a-fdf9-4766-aad9-75627c17c5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:02.111831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1404217a-fdf9-4766-aad9-75627c17c5fc admin admin] PATH_INFO: `/v3/projects/4b0f25b318054d85b6ab8d1bea54aaef` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:02.113262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404217a-fdf9-4766-aad9-75627c17c5fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b0f25b318054d85b6ab8d1bea54aaef)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:02.116564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1404217a-fdf9-4766-aad9-75627c17c5fc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:02.268249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1404217a-fdf9-4766-aad9-75627c17c5fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b0f25b318054d85b6ab8d1bea54aaef', 'request_id': 'req-1404217a-fdf9-4766-aad9-75627c17c5fc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:02.279613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1404217a-fdf9-4766-aad9-75627c17c5fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b0f25b318054d85b6ab8d1bea54aaef', 'request_id': 'req-1404217a-fdf9-4766-aad9-75627c17c5fc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:02.414793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1404217a-fdf9-4766-aad9-75627c17c5fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b0f25b318054d85b6ab8d1bea54aaef', 'request_id': 'req-1404217a-fdf9-4766-aad9-75627c17c5fc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:02.438292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1404217a-fdf9-4766-aad9-75627c17c5fc admin admin] The token cache is being invalidate because project 4b0f25b318054d85b6ab8d1bea54aaef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:02.438590 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1404217a-fdf9-4766-aad9-75627c17c5fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:02.439493 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1404217a-fdf9-4766-aad9-75627c17c5fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:02.441256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 932/1881] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:02 2022] DELETE /identity/v3/projects/4b0f25b318054d85b6ab8d1bea54aaef => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:02.450582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07882e-c523-40d0-b576-c04dbff79cf6 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:02.467179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07882e-c523-40d0-b576-c04dbff79cf6 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:02.538762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cbfc26-fb0a-469c-92a6-364fc24fc8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:02.539528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbfc26-fb0a-469c-92a6-364fc24fc8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:02.539849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbfc26-fb0a-469c-92a6-364fc24fc8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:02.540130 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cbfc26-fb0a-469c-92a6-364fc24fc8eb admin admin] PATH_INFO: `/v3/users/24df618b9abe40bc9f6302451f8b6fc7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:02.541359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbfc26-fb0a-469c-92a6-364fc24fc8eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=24df618b9abe40bc9f6302451f8b6fc7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:02.543349 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cbfc26-fb0a-469c-92a6-364fc24fc8eb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:02.960969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-96cbfc26-fb0a-469c-92a6-364fc24fc8eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24df618b9abe40bc9f6302451f8b6fc7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:03.077969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-96cbfc26-fb0a-469c-92a6-364fc24fc8eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24df618b9abe40bc9f6302451f8b6fc7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:03.079125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404217a-fdf9-4766-aad9-75627c17c5fc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:03.079268 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-96cbfc26-fb0a-469c-92a6-364fc24fc8eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24df618b9abe40bc9f6302451f8b6fc7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:03.080138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-96cbfc26-fb0a-469c-92a6-364fc24fc8eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24df618b9abe40bc9f6302451f8b6fc7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:03.080799 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1404217a-fdf9-4766-aad9-75627c17c5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:03.092519 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-96cbfc26-fb0a-469c-92a6-364fc24fc8eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24df618b9abe40bc9f6302451f8b6fc7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:03.126205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1404217a-fdf9-4766-aad9-75627c17c5fc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:03.137885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 950/1882] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:02 2022] DELETE /identity/v3/users/24df618b9abe40bc9f6302451f8b6fc7 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:03.145804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33a8d9-9a2d-4792-b4e8-de6e1d87cf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:03.145804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33a8d9-9a2d-4792-b4e8-de6e1d87cf63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:03.145804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33a8d9-9a2d-4792-b4e8-de6e1d87cf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:03.145804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c33a8d9-9a2d-4792-b4e8-de6e1d87cf63 admin admin] PATH_INFO: `/v3/users/73fcca66226846339da33af9200857a0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:03.146821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33a8d9-9a2d-4792-b4e8-de6e1d87cf63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=73fcca66226846339da33af9200857a0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:03.148403 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c33a8d9-9a2d-4792-b4e8-de6e1d87cf63 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:03.598667 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1c33a8d9-9a2d-4792-b4e8-de6e1d87cf63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73fcca66226846339da33af9200857a0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:03.708081 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1c33a8d9-9a2d-4792-b4e8-de6e1d87cf63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73fcca66226846339da33af9200857a0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:03.708883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1c33a8d9-9a2d-4792-b4e8-de6e1d87cf63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73fcca66226846339da33af9200857a0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:03.709662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1c33a8d9-9a2d-4792-b4e8-de6e1d87cf63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73fcca66226846339da33af9200857a0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:03.723890 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1c33a8d9-9a2d-4792-b4e8-de6e1d87cf63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73fcca66226846339da33af9200857a0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:03.736837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 933/1883] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:03 2022] DELETE /identity/v3/users/73fcca66226846339da33af9200857a0 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:03.742119 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11518931-8945-4328-bdc6-655151c8b973 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:03.742661 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11518931-8945-4328-bdc6-655151c8b973 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:03.742661 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11518931-8945-4328-bdc6-655151c8b973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:03.765962 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-11518931-8945-4328-bdc6-655151c8b973 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:03.766810 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-11518931-8945-4328-bdc6-655151c8b973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:03.769712 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33a8d9-9a2d-4792-b4e8-de6e1d87cf63 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:03.771833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1c33a8d9-9a2d-4792-b4e8-de6e1d87cf63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:03.846568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 951/1884] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:03.857244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11518931-8945-4328-bdc6-655151c8b973 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:03.858622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-11518931-8945-4328-bdc6-655151c8b973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:03.871176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c33a8d9-9a2d-4792-b4e8-de6e1d87cf63 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:03.877221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869609ee-fd74-48e5-9cec-52a9c566d727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:03.877902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869609ee-fd74-48e5-9cec-52a9c566d727 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:03.878256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869609ee-fd74-48e5-9cec-52a9c566d727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:03.878731 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869609ee-fd74-48e5-9cec-52a9c566d727 admin admin] PATH_INFO: `/v3/projects/02a75d7df1db4133b58088ac3ddac9ea` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:03.880114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869609ee-fd74-48e5-9cec-52a9c566d727 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02a75d7df1db4133b58088ac3ddac9ea)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:03.882603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869609ee-fd74-48e5-9cec-52a9c566d727 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:03.924342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11518931-8945-4328-bdc6-655151c8b973 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:03.928989 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f762439-1650-4833-8c2b-36acfc9c41f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:03.929610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f762439-1650-4833-8c2b-36acfc9c41f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:03.929891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f762439-1650-4833-8c2b-36acfc9c41f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:03.930167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f762439-1650-4833-8c2b-36acfc9c41f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:03.941102 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f762439-1650-4833-8c2b-36acfc9c41f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:03.942532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f762439-1650-4833-8c2b-36acfc9c41f1 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:03.953093 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 952/1885] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:04:04.107672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-869609ee-fd74-48e5-9cec-52a9c566d727 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02a75d7df1db4133b58088ac3ddac9ea', 'request_id': 'req-869609ee-fd74-48e5-9cec-52a9c566d727'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:04.116773 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-869609ee-fd74-48e5-9cec-52a9c566d727 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02a75d7df1db4133b58088ac3ddac9ea', 'request_id': 'req-869609ee-fd74-48e5-9cec-52a9c566d727'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:04.289232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-869609ee-fd74-48e5-9cec-52a9c566d727 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02a75d7df1db4133b58088ac3ddac9ea', 'request_id': 'req-869609ee-fd74-48e5-9cec-52a9c566d727'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:04.289997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:04.301040 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-869609ee-fd74-48e5-9cec-52a9c566d727 admin admin] The token cache is being invalidate because project 02a75d7df1db4133b58088ac3ddac9ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:04.301252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-869609ee-fd74-48e5-9cec-52a9c566d727 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:04.301928 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-869609ee-fd74-48e5-9cec-52a9c566d727 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:04.303507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 934/1886] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:03 2022] DELETE /identity/v3/projects/02a75d7df1db4133b58088ac3ddac9ea => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:04.309365 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2f567-1ecf-4e40-b7b9-ba47a1a027e9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:04.309822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2f567-1ecf-4e40-b7b9-ba47a1a027e9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:04.310247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2f567-1ecf-4e40-b7b9-ba47a1a027e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:04.346714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c2f2f567-1ecf-4e40-b7b9-ba47a1a027e9 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:04.348044 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c2f2f567-1ecf-4e40-b7b9-ba47a1a027e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:04.440609 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 953/1887] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:04.450128 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f5c09-0f1a-4977-9c59-32288985e0ad None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:04.450773 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f5c09-0f1a-4977-9c59-32288985e0ad None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:04.451252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f5c09-0f1a-4977-9c59-32288985e0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:04.479118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-891f5c09-0f1a-4977-9c59-32288985e0ad None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:04.480471 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-891f5c09-0f1a-4977-9c59-32288985e0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:04.488872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 935/1888] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:04.493269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2f567-1ecf-4e40-b7b9-ba47a1a027e9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:04.503937 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2f567-1ecf-4e40-b7b9-ba47a1a027e9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:04.510829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a46da-c0b6-407f-8283-194788b31727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:04.517164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a46da-c0b6-407f-8283-194788b31727 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:04.517164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a46da-c0b6-407f-8283-194788b31727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:04.517164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7a46da-c0b6-407f-8283-194788b31727 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:04.517164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a46da-c0b6-407f-8283-194788b31727 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:04.517164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a46da-c0b6-407f-8283-194788b31727 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:04.517164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7a46da-c0b6-407f-8283-194788b31727 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:04.532240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 954/1889] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:04:04.533266 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f5c09-0f1a-4977-9c59-32288985e0ad None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:04.544078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f5c09-0f1a-4977-9c59-32288985e0ad None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:04.549100 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09c7e5-487c-4f33-8329-f2182d4d084d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:04.549716 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09c7e5-487c-4f33-8329-f2182d4d084d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:04.549998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09c7e5-487c-4f33-8329-f2182d4d084d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:04.550271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab09c7e5-487c-4f33-8329-f2182d4d084d admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:04.550912 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09c7e5-487c-4f33-8329-f2182d4d084d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:04.553380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab09c7e5-487c-4f33-8329-f2182d4d084d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:04.654278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a46da-c0b6-407f-8283-194788b31727 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:04.657165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6a7a46da-c0b6-407f-8283-194788b31727 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:04.679060 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7a46da-c0b6-407f-8283-194788b31727 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:04.690467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac96b92-f9b1-4aac-87b6-098b6053757c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:04.691210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac96b92-f9b1-4aac-87b6-098b6053757c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:04.691476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac96b92-f9b1-4aac-87b6-098b6053757c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:04.691792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac96b92-f9b1-4aac-87b6-098b6053757c admin admin] PATH_INFO: `/v3/projects/13607a27c6334c7b8f067b6c99115d45` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:04.694391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac96b92-f9b1-4aac-87b6-098b6053757c admin admin] RBAC: Authorizing `identity:delete_project(project_id=13607a27c6334c7b8f067b6c99115d45)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:04.695027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac96b92-f9b1-4aac-87b6-098b6053757c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:04.707568 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 936/1890] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:04 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:04.710005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09c7e5-487c-4f33-8329-f2182d4d084d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:04.786549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab09c7e5-487c-4f33-8329-f2182d4d084d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:04.792113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11100d-8cdd-4d71-9c91-7fd19328e78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:04.792718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11100d-8cdd-4d71-9c91-7fd19328e78a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:04.793001 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11100d-8cdd-4d71-9c91-7fd19328e78a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:04.793274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be11100d-8cdd-4d71-9c91-7fd19328e78a admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:04.793906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be11100d-8cdd-4d71-9c91-7fd19328e78a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:04.795900 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be11100d-8cdd-4d71-9c91-7fd19328e78a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:04.905364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9ac96b92-f9b1-4aac-87b6-098b6053757c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13607a27c6334c7b8f067b6c99115d45', 'request_id': 'req-9ac96b92-f9b1-4aac-87b6-098b6053757c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:04.913129 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9ac96b92-f9b1-4aac-87b6-098b6053757c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13607a27c6334c7b8f067b6c99115d45', 'request_id': 'req-9ac96b92-f9b1-4aac-87b6-098b6053757c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:04.940277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 937/1891] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:04 2022] POST /identity/v3/users => generated 287 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:04.944693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11100d-8cdd-4d71-9c91-7fd19328e78a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:05.035961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be11100d-8cdd-4d71-9c91-7fd19328e78a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:05.094722 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb5818-eae6-48c1-b912-957e6832173a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:05.095335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb5818-eae6-48c1-b912-957e6832173a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:05.095614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb5818-eae6-48c1-b912-957e6832173a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:05.095926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecb5818-eae6-48c1-b912-957e6832173a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:05.096593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb5818-eae6-48c1-b912-957e6832173a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:05.098188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecb5818-eae6-48c1-b912-957e6832173a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:05.108645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 938/1892] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:04 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:04:05.112461 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb5818-eae6-48c1-b912-957e6832173a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:05.122818 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecb5818-eae6-48c1-b912-957e6832173a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:05.127539 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96d08a-4fe3-4ad7-9068-26eea9cba1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:05.128155 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96d08a-4fe3-4ad7-9068-26eea9cba1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:05.128439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96d08a-4fe3-4ad7-9068-26eea9cba1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:05.128735 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96d08a-4fe3-4ad7-9068-26eea9cba1e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:05.129333 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96d08a-4fe3-4ad7-9068-26eea9cba1e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:05.130916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96d08a-4fe3-4ad7-9068-26eea9cba1e5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:05.141933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 939/1893] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:04:05.145456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96d08a-4fe3-4ad7-9068-26eea9cba1e5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:05.153537 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9ac96b92-f9b1-4aac-87b6-098b6053757c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13607a27c6334c7b8f067b6c99115d45', 'request_id': 'req-9ac96b92-f9b1-4aac-87b6-098b6053757c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:05.156239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96d08a-4fe3-4ad7-9068-26eea9cba1e5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:05.163792 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8417df-17b1-4228-88cd-0817e2356701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:05.164479 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8417df-17b1-4228-88cd-0817e2356701 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:05.164772 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8417df-17b1-4228-88cd-0817e2356701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:05.165056 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8417df-17b1-4228-88cd-0817e2356701 admin admin] PATH_INFO: `/v3/projects/b1b5fda8372e4b48bf582f3aa506afe3/users/70077796ecd14692b0575342d53d0d56/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:05.167327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9ac96b92-f9b1-4aac-87b6-098b6053757c admin admin] The token cache is being invalidate because project 13607a27c6334c7b8f067b6c99115d45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:05.167622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9ac96b92-f9b1-4aac-87b6-098b6053757c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:05.168341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9ac96b92-f9b1-4aac-87b6-098b6053757c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:05.169898 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 955/1894] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:04 2022] DELETE /identity/v3/projects/13607a27c6334c7b8f067b6c99115d45 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:05.173907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac96b92-f9b1-4aac-87b6-098b6053757c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:05.175288 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9ac96b92-f9b1-4aac-87b6-098b6053757c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:05.181740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8417df-17b1-4228-88cd-0817e2356701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b5fda8372e4b48bf582f3aa506afe3, user_id=70077796ecd14692b0575342d53d0d56, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:05.184620 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8417df-17b1-4228-88cd-0817e2356701 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:05.189039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac96b92-f9b1-4aac-87b6-098b6053757c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:05.193692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34561341-72fc-4cfc-b19a-8e852882c9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:05.194309 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34561341-72fc-4cfc-b19a-8e852882c9ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:05.194584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34561341-72fc-4cfc-b19a-8e852882c9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:05.194852 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34561341-72fc-4cfc-b19a-8e852882c9ac admin admin] PATH_INFO: `/v3/users/272e5409da6a47f692fc4befd98c4c4e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:05.197865 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34561341-72fc-4cfc-b19a-8e852882c9ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=272e5409da6a47f692fc4befd98c4c4e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:05.199797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34561341-72fc-4cfc-b19a-8e852882c9ac admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:05.274921 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 940/1895] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:05 2022] PUT /identity/v3/projects/b1b5fda8372e4b48bf582f3aa506afe3/users/70077796ecd14692b0575342d53d0d56/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:05.285391 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d5d3e-4af0-4927-ae61-0632af96dc21 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:05.285677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d5d3e-4af0-4927-ae61-0632af96dc21 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:05.285949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008d5d3e-4af0-4927-ae61-0632af96dc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:05.319776 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-008d5d3e-4af0-4927-ae61-0632af96dc21 None None] MFA Rules not processed for user `70077796ecd14692b0575342d53d0d56`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:05.321142 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-008d5d3e-4af0-4927-ae61-0632af96dc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:05.389167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 941/1896] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:05.407806 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d5d3e-4af0-4927-ae61-0632af96dc21 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:05.409414 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-008d5d3e-4af0-4927-ae61-0632af96dc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:05.440149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34561341-72fc-4cfc-b19a-8e852882c9ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '272e5409da6a47f692fc4befd98c4c4e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:05.480903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008d5d3e-4af0-4927-ae61-0632af96dc21 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:05.485693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8235d-80fe-4c5b-be5d-4b95ad8bd4b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:05.485693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8235d-80fe-4c5b-be5d-4b95ad8bd4b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:05.485693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8235d-80fe-4c5b-be5d-4b95ad8bd4b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:05.486110 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8235d-80fe-4c5b-be5d-4b95ad8bd4b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:05.498771 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8235d-80fe-4c5b-be5d-4b95ad8bd4b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:05.500503 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8235d-80fe-4c5b-be5d-4b95ad8bd4b3 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:05.515860 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 942/1897] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:05.535766 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34561341-72fc-4cfc-b19a-8e852882c9ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '272e5409da6a47f692fc4befd98c4c4e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:05.541790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34561341-72fc-4cfc-b19a-8e852882c9ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '272e5409da6a47f692fc4befd98c4c4e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:05.541790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34561341-72fc-4cfc-b19a-8e852882c9ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '272e5409da6a47f692fc4befd98c4c4e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:05.545741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8235d-80fe-4c5b-be5d-4b95ad8bd4b3 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:05.546852 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-34561341-72fc-4cfc-b19a-8e852882c9ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '272e5409da6a47f692fc4befd98c4c4e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:05.547294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-75e8235d-80fe-4c5b-be5d-4b95ad8bd4b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:05.567313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 956/1898] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:05 2022] DELETE /identity/v3/users/272e5409da6a47f692fc4befd98c4c4e => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:05.644335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8235d-80fe-4c5b-be5d-4b95ad8bd4b3 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:05.734937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3dfaa-a3f0-4abb-871f-ee692e1ad04f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:05.735831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3dfaa-a3f0-4abb-871f-ee692e1ad04f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:05.738748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3dfaa-a3f0-4abb-871f-ee692e1ad04f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:05.738748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3dfaa-a3f0-4abb-871f-ee692e1ad04f admin admin] PATH_INFO: `/v3/users/70077796ecd14692b0575342d53d0d56` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:05.738748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3dfaa-a3f0-4abb-871f-ee692e1ad04f admin admin] RBAC: Authorizing `identity:delete_user(user_id=70077796ecd14692b0575342d53d0d56)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:05.739627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3dfaa-a3f0-4abb-871f-ee692e1ad04f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:06.118708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3f3dfaa-a3f0-4abb-871f-ee692e1ad04f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70077796ecd14692b0575342d53d0d56'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:06.163043 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34561341-72fc-4cfc-b19a-8e852882c9ac admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:06.164956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-34561341-72fc-4cfc-b19a-8e852882c9ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:06.184201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34561341-72fc-4cfc-b19a-8e852882c9ac admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:06.190759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa682600-5e0a-4f82-84b0-2016979d47ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:06.193033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa682600-5e0a-4f82-84b0-2016979d47ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:06.195015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa682600-5e0a-4f82-84b0-2016979d47ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:06.196223 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa682600-5e0a-4f82-84b0-2016979d47ca admin admin] PATH_INFO: `/v3/projects/0cce71b499c84857b5993fd0de8c4c79` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:06.198239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa682600-5e0a-4f82-84b0-2016979d47ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cce71b499c84857b5993fd0de8c4c79)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:06.203265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa682600-5e0a-4f82-84b0-2016979d47ca admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:06.209711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3f3dfaa-a3f0-4abb-871f-ee692e1ad04f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70077796ecd14692b0575342d53d0d56'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:06.211433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3f3dfaa-a3f0-4abb-871f-ee692e1ad04f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70077796ecd14692b0575342d53d0d56'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:06.212824 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3f3dfaa-a3f0-4abb-871f-ee692e1ad04f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70077796ecd14692b0575342d53d0d56'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:06.240790 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e3f3dfaa-a3f0-4abb-871f-ee692e1ad04f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70077796ecd14692b0575342d53d0d56'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:06.257351 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 943/1899] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:05 2022] DELETE /identity/v3/users/70077796ecd14692b0575342d53d0d56 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:06.262277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f4d9e-6a25-4920-91b9-1004cb049c67 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:06.262569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f4d9e-6a25-4920-91b9-1004cb049c67 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:06.262847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f4d9e-6a25-4920-91b9-1004cb049c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:06.282230 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6c8f4d9e-6a25-4920-91b9-1004cb049c67 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:06.283447 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6c8f4d9e-6a25-4920-91b9-1004cb049c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:06.350596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 944/1900] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:06.360139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f4d9e-6a25-4920-91b9-1004cb049c67 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:06.362005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6c8f4d9e-6a25-4920-91b9-1004cb049c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:06.366950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-aa682600-5e0a-4f82-84b0-2016979d47ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cce71b499c84857b5993fd0de8c4c79', 'request_id': 'req-aa682600-5e0a-4f82-84b0-2016979d47ca'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:06.376286 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-aa682600-5e0a-4f82-84b0-2016979d47ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cce71b499c84857b5993fd0de8c4c79', 'request_id': 'req-aa682600-5e0a-4f82-84b0-2016979d47ca'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:06.423609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f4d9e-6a25-4920-91b9-1004cb049c67 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:06.477346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa5bb2-79cc-4e56-80fa-b0fc853637ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:06.478149 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa5bb2-79cc-4e56-80fa-b0fc853637ce service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:06.478526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa5bb2-79cc-4e56-80fa-b0fc853637ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:06.478965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa5bb2-79cc-4e56-80fa-b0fc853637ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:06.541768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa5bb2-79cc-4e56-80fa-b0fc853637ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:06.543105 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fa5bb2-79cc-4e56-80fa-b0fc853637ce service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:06.553324 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 945/1901] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:04:06.586239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-aa682600-5e0a-4f82-84b0-2016979d47ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cce71b499c84857b5993fd0de8c4c79', 'request_id': 'req-aa682600-5e0a-4f82-84b0-2016979d47ca'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:06.602839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-aa682600-5e0a-4f82-84b0-2016979d47ca admin admin] The token cache is being invalidate because project 0cce71b499c84857b5993fd0de8c4c79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:06.603414 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-aa682600-5e0a-4f82-84b0-2016979d47ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:06.604228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-aa682600-5e0a-4f82-84b0-2016979d47ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:06.605819 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 957/1902] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:06 2022] DELETE /identity/v3/projects/0cce71b499c84857b5993fd0de8c4c79 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:06.611087 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f724f-0c94-47d3-b06e-382e39cee2f3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:06.611453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f724f-0c94-47d3-b06e-382e39cee2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:06.611759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258f724f-0c94-47d3-b06e-382e39cee2f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:06.633506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-258f724f-0c94-47d3-b06e-382e39cee2f3 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:06.635331 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-258f724f-0c94-47d3-b06e-382e39cee2f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:06.644486 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 946/1903] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:06.652584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa9bf6-6633-4e2c-bc92-dd0c321e0c64 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:06.653165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa9bf6-6633-4e2c-bc92-dd0c321e0c64 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:06.653533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa9bf6-6633-4e2c-bc92-dd0c321e0c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:06.676151 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1efa9bf6-6633-4e2c-bc92-dd0c321e0c64 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:06.677447 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1efa9bf6-6633-4e2c-bc92-dd0c321e0c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:06.686265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 958/1904] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:06.690641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f724f-0c94-47d3-b06e-382e39cee2f3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:06.703517 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258f724f-0c94-47d3-b06e-382e39cee2f3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:06.719993 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f242baa-cde5-4163-b186-6afadad5f1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:06.719993 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f242baa-cde5-4163-b186-6afadad5f1b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:06.719993 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f242baa-cde5-4163-b186-6afadad5f1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:06.719993 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f242baa-cde5-4163-b186-6afadad5f1b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:06.721865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f242baa-cde5-4163-b186-6afadad5f1b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:06.721865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f242baa-cde5-4163-b186-6afadad5f1b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:06.721865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f242baa-cde5-4163-b186-6afadad5f1b4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:06.729779 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 947/1905] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:04:06.734814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa9bf6-6633-4e2c-bc92-dd0c321e0c64 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:06.759902 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa9bf6-6633-4e2c-bc92-dd0c321e0c64 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:06.768908 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd10e6-7868-4c67-902a-7adf7b5fd418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:06.996179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd10e6-7868-4c67-902a-7adf7b5fd418 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:06.996179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd10e6-7868-4c67-902a-7adf7b5fd418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:06.996179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd10e6-7868-4c67-902a-7adf7b5fd418 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:06.996179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd10e6-7868-4c67-902a-7adf7b5fd418 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:06.996179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd10e6-7868-4c67-902a-7adf7b5fd418 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:07.179886 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 959/1906] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:06 2022] POST /identity/v3/projects => generated 385 bytes in 446 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:07.184285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f242baa-cde5-4163-b186-6afadad5f1b4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:07.194945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f242baa-cde5-4163-b186-6afadad5f1b4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:07.296090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea90b6d-b7cd-4028-9457-d96ebf9b7b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:07.297522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea90b6d-b7cd-4028-9457-d96ebf9b7b49 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:07.298296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea90b6d-b7cd-4028-9457-d96ebf9b7b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:07.298887 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea90b6d-b7cd-4028-9457-d96ebf9b7b49 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:07.300194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea90b6d-b7cd-4028-9457-d96ebf9b7b49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:07.304835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea90b6d-b7cd-4028-9457-d96ebf9b7b49 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:07.315718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:07.556337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 948/1907] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:07 2022] POST /identity/v3/users => generated 293 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:07.561824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd10e6-7868-4c67-902a-7adf7b5fd418 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:07.566009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-99dd10e6-7868-4c67-902a-7adf7b5fd418 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:07.576342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea90b6d-b7cd-4028-9457-d96ebf9b7b49 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:07.615463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea90b6d-b7cd-4028-9457-d96ebf9b7b49 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:07.615909 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd10e6-7868-4c67-902a-7adf7b5fd418 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:07.616176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373d3e6-f60e-4998-876f-723855d10312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:07.616176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373d3e6-f60e-4998-876f-723855d10312 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:07.616351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7a1c9-7e78-4f0a-a8a0-8a07243aacc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:07.616448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373d3e6-f60e-4998-876f-723855d10312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:07.616448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373d3e6-f60e-4998-876f-723855d10312 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:07.616615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7a1c9-7e78-4f0a-a8a0-8a07243aacc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:07.616615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7a1c9-7e78-4f0a-a8a0-8a07243aacc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:07.616615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7a1c9-7e78-4f0a-a8a0-8a07243aacc9 admin admin] PATH_INFO: `/v3/projects/b1b5fda8372e4b48bf582f3aa506afe3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:07.616843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373d3e6-f60e-4998-876f-723855d10312 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:07.616934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7a1c9-7e78-4f0a-a8a0-8a07243aacc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b5fda8372e4b48bf582f3aa506afe3)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:07.617016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6373d3e6-f60e-4998-876f-723855d10312 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:07.617101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7a1c9-7e78-4f0a-a8a0-8a07243aacc9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:07.625484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 949/1908] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:07.629542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373d3e6-f60e-4998-876f-723855d10312 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:07.640334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6373d3e6-f60e-4998-876f-723855d10312 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:07.647028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcfe6b-a1de-431e-b2d7-453f5f3299fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:07.647847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcfe6b-a1de-431e-b2d7-453f5f3299fb admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:07.648131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcfe6b-a1de-431e-b2d7-453f5f3299fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:07.648402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bcfe6b-a1de-431e-b2d7-453f5f3299fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:07.649034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcfe6b-a1de-431e-b2d7-453f5f3299fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:07.650676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bcfe6b-a1de-431e-b2d7-453f5f3299fb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:07.660913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 950/1909] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:04:07.665571 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcfe6b-a1de-431e-b2d7-453f5f3299fb admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:07.675078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bcfe6b-a1de-431e-b2d7-453f5f3299fb admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:07.679851 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e87b14-8fb1-4a8b-a376-4b78e7f9c696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:07.680521 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e87b14-8fb1-4a8b-a376-4b78e7f9c696 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:07.680811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e87b14-8fb1-4a8b-a376-4b78e7f9c696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:07.681086 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e87b14-8fb1-4a8b-a376-4b78e7f9c696 admin admin] PATH_INFO: `/v3/projects/df294c19c07942039b3631e28b25164e/users/bf91933f429c4148adb053b58c079b2b/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:07.697358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e87b14-8fb1-4a8b-a376-4b78e7f9c696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df294c19c07942039b3631e28b25164e, user_id=bf91933f429c4148adb053b58c079b2b, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:07.700045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e87b14-8fb1-4a8b-a376-4b78e7f9c696 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:07.914728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 951/1910] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:07 2022] PUT /identity/v3/projects/df294c19c07942039b3631e28b25164e/users/bf91933f429c4148adb053b58c079b2b/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:07.922661 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-19a7a1c9-7e78-4f0a-a8a0-8a07243aacc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b5fda8372e4b48bf582f3aa506afe3', 'request_id': 'req-19a7a1c9-7e78-4f0a-a8a0-8a07243aacc9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:07.925048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69715c0-164a-47a2-91db-e2a02e618072 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:07.925338 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69715c0-164a-47a2-91db-e2a02e618072 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:07.925644 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69715c0-164a-47a2-91db-e2a02e618072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:07.929856 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-19a7a1c9-7e78-4f0a-a8a0-8a07243aacc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b5fda8372e4b48bf582f3aa506afe3', 'request_id': 'req-19a7a1c9-7e78-4f0a-a8a0-8a07243aacc9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:07.954401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f69715c0-164a-47a2-91db-e2a02e618072 None None] MFA Rules not processed for user `bf91933f429c4148adb053b58c079b2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:07.955625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f69715c0-164a-47a2-91db-e2a02e618072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:08.028412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 952/1911] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:08.032805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69715c0-164a-47a2-91db-e2a02e618072 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:08.073903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-19a7a1c9-7e78-4f0a-a8a0-8a07243aacc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b5fda8372e4b48bf582f3aa506afe3', 'request_id': 'req-19a7a1c9-7e78-4f0a-a8a0-8a07243aacc9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:08.087261 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-19a7a1c9-7e78-4f0a-a8a0-8a07243aacc9 admin admin] The token cache is being invalidate because project b1b5fda8372e4b48bf582f3aa506afe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:08.087598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-19a7a1c9-7e78-4f0a-a8a0-8a07243aacc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:08.088358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-19a7a1c9-7e78-4f0a-a8a0-8a07243aacc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:08.090232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 960/1912] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:07 2022] DELETE /identity/v3/projects/b1b5fda8372e4b48bf582f3aa506afe3 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:08.095114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed171f-b347-4e30-a5d9-60cac8638a92 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.095534 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed171f-b347-4e30-a5d9-60cac8638a92 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.096274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed171f-b347-4e30-a5d9-60cac8638a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.110099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69715c0-164a-47a2-91db-e2a02e618072 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:08.120288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ebd45-f201-4caa-92c0-cc9e604e873c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:08.120288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ebd45-f201-4caa-92c0-cc9e604e873c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.120288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ebd45-f201-4caa-92c0-cc9e604e873c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.124169 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63ebd45-f201-4caa-92c0-cc9e604e873c admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.124169 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ebd45-f201-4caa-92c0-cc9e604e873c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:08.124169 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63ebd45-f201-4caa-92c0-cc9e604e873c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:08.129407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-99ed171f-b347-4e30-a5d9-60cac8638a92 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:08.131234 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-99ed171f-b347-4e30-a5d9-60cac8638a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:08.140748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 961/1913] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:08.150166 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e84046-903e-4b8e-82bc-e1914c1b09a0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.150532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e84046-903e-4b8e-82bc-e1914c1b09a0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.150880 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e84046-903e-4b8e-82bc-e1914c1b09a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.171252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-99e84046-903e-4b8e-82bc-e1914c1b09a0 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:08.172486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-99e84046-903e-4b8e-82bc-e1914c1b09a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:08.182160 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 962/1914] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:08.187280 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e84046-903e-4b8e-82bc-e1914c1b09a0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:08.199351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e84046-903e-4b8e-82bc-e1914c1b09a0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:08.205159 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6d4ec-e046-4cb5-aee2-ca03f89812de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:08.207310 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6d4ec-e046-4cb5-aee2-ca03f89812de admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.207310 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6d4ec-e046-4cb5-aee2-ca03f89812de admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.207310 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a6d4ec-e046-4cb5-aee2-ca03f89812de admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.207310 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6d4ec-e046-4cb5-aee2-ca03f89812de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:08.207310 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6d4ec-e046-4cb5-aee2-ca03f89812de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:08.209107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a6d4ec-e046-4cb5-aee2-ca03f89812de admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:08.228816 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 963/1915] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:04:08.236421 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6d4ec-e046-4cb5-aee2-ca03f89812de admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:08.266102 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a6d4ec-e046-4cb5-aee2-ca03f89812de admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:08.271491 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997ecb9-ce35-4750-ad45-9cf2798cabdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:08.272449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997ecb9-ce35-4750-ad45-9cf2798cabdc admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.272916 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997ecb9-ce35-4750-ad45-9cf2798cabdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.273880 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4997ecb9-ce35-4750-ad45-9cf2798cabdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.274527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997ecb9-ce35-4750-ad45-9cf2798cabdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:08.276479 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4997ecb9-ce35-4750-ad45-9cf2798cabdc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:08.281531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 953/1916] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:08 2022] POST /identity/v3/projects => generated 385 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:08.285657 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ebd45-f201-4caa-92c0-cc9e604e873c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:08.288263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e63ebd45-f201-4caa-92c0-cc9e604e873c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:08.357969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63ebd45-f201-4caa-92c0-cc9e604e873c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:08.362788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b1fdd-b975-4cc4-a70f-cb6b354d62e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:08.363415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b1fdd-b975-4cc4-a70f-cb6b354d62e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.363865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b1fdd-b975-4cc4-a70f-cb6b354d62e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.364034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9b1fdd-b975-4cc4-a70f-cb6b354d62e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.364950 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b1fdd-b975-4cc4-a70f-cb6b354d62e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:08.367260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9b1fdd-b975-4cc4-a70f-cb6b354d62e6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:08.418914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 964/1917] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:08 2022] POST /identity/v3/projects => generated 369 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:08.423322 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997ecb9-ce35-4750-ad45-9cf2798cabdc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:08.485967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4997ecb9-ce35-4750-ad45-9cf2798cabdc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:08.490867 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae893b3-2b71-4f3b-b0f3-1c02656573af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:08.491476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae893b3-2b71-4f3b-b0f3-1c02656573af admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.491791 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae893b3-2b71-4f3b-b0f3-1c02656573af admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.492068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae893b3-2b71-4f3b-b0f3-1c02656573af admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.492695 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae893b3-2b71-4f3b-b0f3-1c02656573af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:08.494626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae893b3-2b71-4f3b-b0f3-1c02656573af admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:08.537401 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 954/1918] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:08 2022] POST /identity/v3/users => generated 299 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:08.541419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b1fdd-b975-4cc4-a70f-cb6b354d62e6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:08.551628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9b1fdd-b975-4cc4-a70f-cb6b354d62e6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:08.556457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67ef87-cd02-4acc-b77f-6642b3923d57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:08.557085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67ef87-cd02-4acc-b77f-6642b3923d57 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.557369 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67ef87-cd02-4acc-b77f-6642b3923d57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.557645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67ef87-cd02-4acc-b77f-6642b3923d57 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.558284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67ef87-cd02-4acc-b77f-6642b3923d57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:08.559912 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67ef87-cd02-4acc-b77f-6642b3923d57 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:08.572399 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 955/1919] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:04:08.576585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67ef87-cd02-4acc-b77f-6642b3923d57 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:08.587857 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67ef87-cd02-4acc-b77f-6642b3923d57 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:08.593124 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e5094-b7aa-4ad9-8913-5b031e6a511a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:08.599708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e5094-b7aa-4ad9-8913-5b031e6a511a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.599708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e5094-b7aa-4ad9-8913-5b031e6a511a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.599708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74e5094-b7aa-4ad9-8913-5b031e6a511a admin admin] PATH_INFO: `/v3/projects/4a460a328246402a93a8e724e545632a/users/b5111d51d8264a7980ee0dd70d1b5282/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.611711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e5094-b7aa-4ad9-8913-5b031e6a511a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a460a328246402a93a8e724e545632a, user_id=b5111d51d8264a7980ee0dd70d1b5282, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:08.614523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74e5094-b7aa-4ad9-8913-5b031e6a511a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:08.633127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 965/1920] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:08 2022] POST /identity/v3/users => generated 285 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:08.637683 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae893b3-2b71-4f3b-b0f3-1c02656573af admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:08.648170 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae893b3-2b71-4f3b-b0f3-1c02656573af admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:08.653171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef561fa4-44f4-4b7b-8c6e-81dbd80f8478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:08.653774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef561fa4-44f4-4b7b-8c6e-81dbd80f8478 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.654056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef561fa4-44f4-4b7b-8c6e-81dbd80f8478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.654338 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef561fa4-44f4-4b7b-8c6e-81dbd80f8478 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.655174 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef561fa4-44f4-4b7b-8c6e-81dbd80f8478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:08.656720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef561fa4-44f4-4b7b-8c6e-81dbd80f8478 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:08.667165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 966/1921] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:04:08.671778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef561fa4-44f4-4b7b-8c6e-81dbd80f8478 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:08.682252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef561fa4-44f4-4b7b-8c6e-81dbd80f8478 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:08.687614 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208cbdb-ced7-4ed6-85d0-6d36e832cc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:08.688214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208cbdb-ced7-4ed6-85d0-6d36e832cc6a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.688500 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208cbdb-ced7-4ed6-85d0-6d36e832cc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.688778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208cbdb-ced7-4ed6-85d0-6d36e832cc6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.689411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208cbdb-ced7-4ed6-85d0-6d36e832cc6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:08.691102 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208cbdb-ced7-4ed6-85d0-6d36e832cc6a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:08.702983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 967/1922] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:04:08.707029 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208cbdb-ced7-4ed6-85d0-6d36e832cc6a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:08.717270 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208cbdb-ced7-4ed6-85d0-6d36e832cc6a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:08.722336 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23145c21-cfa2-4450-997f-fc5dd49f1201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:08.723110 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23145c21-cfa2-4450-997f-fc5dd49f1201 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.723541 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23145c21-cfa2-4450-997f-fc5dd49f1201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.723858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23145c21-cfa2-4450-997f-fc5dd49f1201 admin admin] PATH_INFO: `/v3/projects/7798d7c788f04d099151350cf2fb1097/users/e71afa4b7ff24c86aacdeb2b4a3d73f6/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.738939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23145c21-cfa2-4450-997f-fc5dd49f1201 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7798d7c788f04d099151350cf2fb1097, user_id=e71afa4b7ff24c86aacdeb2b4a3d73f6, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:08.742323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23145c21-cfa2-4450-997f-fc5dd49f1201 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:08.742872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 956/1923] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:08 2022] PUT /identity/v3/projects/4a460a328246402a93a8e724e545632a/users/b5111d51d8264a7980ee0dd70d1b5282/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:08.746831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e5094-b7aa-4ad9-8913-5b031e6a511a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:08.812513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74e5094-b7aa-4ad9-8913-5b031e6a511a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:08.817347 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39d8fb-6a55-438e-801e-dbe39c46d801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:08.818571 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39d8fb-6a55-438e-801e-dbe39c46d801 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.819029 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39d8fb-6a55-438e-801e-dbe39c46d801 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.819319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a39d8fb-6a55-438e-801e-dbe39c46d801 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.819996 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a39d8fb-6a55-438e-801e-dbe39c46d801 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:08.821601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a39d8fb-6a55-438e-801e-dbe39c46d801 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:08.831753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 957/1924] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:04:08.836852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39d8fb-6a55-438e-801e-dbe39c46d801 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:08.846727 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a39d8fb-6a55-438e-801e-dbe39c46d801 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:08.851763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628674f-1aa8-450b-9e81-0e93e1ae95aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:08.852412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628674f-1aa8-450b-9e81-0e93e1ae95aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.852702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628674f-1aa8-450b-9e81-0e93e1ae95aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.852979 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628674f-1aa8-450b-9e81-0e93e1ae95aa admin admin] PATH_INFO: `/v3/projects/4a460a328246402a93a8e724e545632a/users/b5111d51d8264a7980ee0dd70d1b5282/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.854953 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628674f-1aa8-450b-9e81-0e93e1ae95aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a460a328246402a93a8e724e545632a, user_id=b5111d51d8264a7980ee0dd70d1b5282, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:08.857619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628674f-1aa8-450b-9e81-0e93e1ae95aa admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:08.872365 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 958/1925] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:08 2022] PUT /identity/v3/projects/4a460a328246402a93a8e724e545632a/users/b5111d51d8264a7980ee0dd70d1b5282/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:08.884861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d6bc4-bdb3-4607-ac32-41ee5975daab None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.885440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d6bc4-bdb3-4607-ac32-41ee5975daab None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.885724 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d6bc4-bdb3-4607-ac32-41ee5975daab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.913546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c60d6bc4-bdb3-4607-ac32-41ee5975daab None None] MFA Rules not processed for user `b5111d51d8264a7980ee0dd70d1b5282`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:08.914772 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c60d6bc4-bdb3-4607-ac32-41ee5975daab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:08.931088 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 968/1926] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:08 2022] PUT /identity/v3/projects/7798d7c788f04d099151350cf2fb1097/users/e71afa4b7ff24c86aacdeb2b4a3d73f6/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:08.941340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b25510-91cc-43cf-91ee-c9e1b7bdf237 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:08.941627 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b25510-91cc-43cf-91ee-c9e1b7bdf237 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:08.941909 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b25510-91cc-43cf-91ee-c9e1b7bdf237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:08.971115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-10b25510-91cc-43cf-91ee-c9e1b7bdf237 None None] MFA Rules not processed for user `e71afa4b7ff24c86aacdeb2b4a3d73f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:08.972412 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-10b25510-91cc-43cf-91ee-c9e1b7bdf237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:08.993650 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:08.994500 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 959/1927] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:08 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:09.010899 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d6bc4-bdb3-4607-ac32-41ee5975daab None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:09.010899 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c60d6bc4-bdb3-4607-ac32-41ee5975daab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:09.053269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 969/1928] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:09.065849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b25510-91cc-43cf-91ee-c9e1b7bdf237 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:09.067291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-10b25510-91cc-43cf-91ee-c9e1b7bdf237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:09.069450 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60d6bc4-bdb3-4607-ac32-41ee5975daab None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:09.119112 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e0df7d-796d-4fd3-bf4d-b5794238d66c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:09.119781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0df7d-796d-4fd3-bf4d-b5794238d66c service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:09.120109 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0df7d-796d-4fd3-bf4d-b5794238d66c service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:09.120392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e0df7d-796d-4fd3-bf4d-b5794238d66c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:09.151027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b25510-91cc-43cf-91ee-c9e1b7bdf237 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:09.159297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45956f-09e8-4842-82f6-90275bd8268b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6dee005fc3d4fb593a1771e7851c20a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:09.159297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45956f-09e8-4842-82f6-90275bd8268b service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:09.159297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45956f-09e8-4842-82f6-90275bd8268b service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:09.159297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45956f-09e8-4842-82f6-90275bd8268b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:09.165220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45956f-09e8-4842-82f6-90275bd8268b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:09.166671 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45956f-09e8-4842-82f6-90275bd8268b service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:09.175967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 970/1929] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:04:09.193391 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0df7d-796d-4fd3-bf4d-b5794238d66c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:09.196076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e0df7d-796d-4fd3-bf4d-b5794238d66c service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:09.212470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 960/1930] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:09 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:04:10.088850 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c4a3b-c7be-4606-b1aa-93a2f369e83b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:10.089351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c4a3b-c7be-4606-b1aa-93a2f369e83b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:10.089430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c4a3b-c7be-4606-b1aa-93a2f369e83b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:10.091129 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 971/1931] 10.222.0.67 () {54 vars in 921 bytes} [Mon Sep 26 18:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:04:10.101844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098944c5-db35-434a-a546-f5e3b4948c32 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:10.101844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098944c5-db35-434a-a546-f5e3b4948c32 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:10.101844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098944c5-db35-434a-a546-f5e3b4948c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:10.125468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-098944c5-db35-434a-a546-f5e3b4948c32 None None] MFA Rules not processed for user `b9ed29a3f6be4d64bb3751f9e95eb56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:10.129415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-098944c5-db35-434a-a546-f5e3b4948c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:10.227006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 961/1932] 10.222.0.67 () {58 vars in 1018 bytes} [Mon Sep 26 18:04:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:10.283240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c4a3b-c7be-4606-b1aa-93a2f369e83b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:10.285577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4e6c4a3b-c7be-4606-b1aa-93a2f369e83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:10.310902 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c4a3b-c7be-4606-b1aa-93a2f369e83b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:10.320431 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c2e74-c3e9-485b-8175-11c278e3fc1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:10.321057 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c2e74-c3e9-485b-8175-11c278e3fc1f service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:10.321338 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c2e74-c3e9-485b-8175-11c278e3fc1f service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:10.321614 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c2e74-c3e9-485b-8175-11c278e3fc1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:10.332555 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c2e74-c3e9-485b-8175-11c278e3fc1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:10.334806 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c2e74-c3e9-485b-8175-11c278e3fc1f service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:10.346965 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 972/1933] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:10.526175 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:11.214487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098944c5-db35-434a-a546-f5e3b4948c32 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:11.288730 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098944c5-db35-434a-a546-f5e3b4948c32 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:11.293452 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9731c6-eb64-48ee-8d0f-f8aacb89f679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:11.294178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9731c6-eb64-48ee-8d0f-f8aacb89f679 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:11.294467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9731c6-eb64-48ee-8d0f-f8aacb89f679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:11.294746 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9731c6-eb64-48ee-8d0f-f8aacb89f679 admin admin] PATH_INFO: `/v3/users/bf91933f429c4148adb053b58c079b2b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:11.295946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9731c6-eb64-48ee-8d0f-f8aacb89f679 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf91933f429c4148adb053b58c079b2b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:11.297902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9731c6-eb64-48ee-8d0f-f8aacb89f679 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:11.548806 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-aa9731c6-eb64-48ee-8d0f-f8aacb89f679 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf91933f429c4148adb053b58c079b2b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:11.585957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:11.735382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-aa9731c6-eb64-48ee-8d0f-f8aacb89f679 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf91933f429c4148adb053b58c079b2b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:11.736241 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-aa9731c6-eb64-48ee-8d0f-f8aacb89f679 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf91933f429c4148adb053b58c079b2b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:11.737076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-aa9731c6-eb64-48ee-8d0f-f8aacb89f679 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf91933f429c4148adb053b58c079b2b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:11.751853 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-aa9731c6-eb64-48ee-8d0f-f8aacb89f679 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf91933f429c4148adb053b58c079b2b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:11.765520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 962/1934] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:11 2022] DELETE /identity/v3/users/bf91933f429c4148adb053b58c079b2b => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:11.771673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1fbcd-a9a8-474c-8342-cff3da2ba7fb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:11.772250 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1fbcd-a9a8-474c-8342-cff3da2ba7fb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:11.772809 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1fbcd-a9a8-474c-8342-cff3da2ba7fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:11.794587 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6ff1fbcd-a9a8-474c-8342-cff3da2ba7fb None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:11.795936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6ff1fbcd-a9a8-474c-8342-cff3da2ba7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:11.864071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 973/1935] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:11.874765 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9731c6-eb64-48ee-8d0f-f8aacb89f679 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:11.876229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-aa9731c6-eb64-48ee-8d0f-f8aacb89f679 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:11.943208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9731c6-eb64-48ee-8d0f-f8aacb89f679 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:11.950325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca0503-4356-4791-b078-ffb96b6deeb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:11.950947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca0503-4356-4791-b078-ffb96b6deeb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:11.951232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca0503-4356-4791-b078-ffb96b6deeb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:11.951573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca0503-4356-4791-b078-ffb96b6deeb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:11.962274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca0503-4356-4791-b078-ffb96b6deeb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:11.963933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cca0503-4356-4791-b078-ffb96b6deeb8 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:11.975087 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 963/1936] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:14.119459 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:14.124873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1fbcd-a9a8-474c-8342-cff3da2ba7fb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:14.127493 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6ff1fbcd-a9a8-474c-8342-cff3da2ba7fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:14.174562 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1fbcd-a9a8-474c-8342-cff3da2ba7fb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:14.180488 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed7463-b665-446e-a3ca-37ede497a4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:14.181582 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7463-b665-446e-a3ca-37ede497a4c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:14.181870 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7463-b665-446e-a3ca-37ede497a4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:14.182241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed7463-b665-446e-a3ca-37ede497a4c6 admin admin] PATH_INFO: `/v3/projects/df294c19c07942039b3631e28b25164e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:14.184436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed7463-b665-446e-a3ca-37ede497a4c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df294c19c07942039b3631e28b25164e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:14.186914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed7463-b665-446e-a3ca-37ede497a4c6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:14.481218 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a5ed7463-b665-446e-a3ca-37ede497a4c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df294c19c07942039b3631e28b25164e', 'request_id': 'req-a5ed7463-b665-446e-a3ca-37ede497a4c6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:14.487458 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a5ed7463-b665-446e-a3ca-37ede497a4c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df294c19c07942039b3631e28b25164e', 'request_id': 'req-a5ed7463-b665-446e-a3ca-37ede497a4c6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:14.882151 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a5ed7463-b665-446e-a3ca-37ede497a4c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df294c19c07942039b3631e28b25164e', 'request_id': 'req-a5ed7463-b665-446e-a3ca-37ede497a4c6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:14.900787 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a5ed7463-b665-446e-a3ca-37ede497a4c6 admin admin] The token cache is being invalidate because project df294c19c07942039b3631e28b25164e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:14.901084 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a5ed7463-b665-446e-a3ca-37ede497a4c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:14.903490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a5ed7463-b665-446e-a3ca-37ede497a4c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:14.905404 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 974/1937] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:14 2022] DELETE /identity/v3/projects/df294c19c07942039b3631e28b25164e => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:14.909496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca0503-4356-4791-b078-ffb96b6deeb8 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:14.929068 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca0503-4356-4791-b078-ffb96b6deeb8 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:15.054977 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e3862-b25b-4980-88b8-98f0b3b9029f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:15.056180 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e3862-b25b-4980-88b8-98f0b3b9029f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:15.056180 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e3862-b25b-4980-88b8-98f0b3b9029f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:15.056180 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2e3862-b25b-4980-88b8-98f0b3b9029f admin admin] PATH_INFO: `/v3/users/b5111d51d8264a7980ee0dd70d1b5282` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:15.058204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e3862-b25b-4980-88b8-98f0b3b9029f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5111d51d8264a7980ee0dd70d1b5282)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:15.064817 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2e3862-b25b-4980-88b8-98f0b3b9029f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:15.357600 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9e2e3862-b25b-4980-88b8-98f0b3b9029f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5111d51d8264a7980ee0dd70d1b5282'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:15.552672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9e2e3862-b25b-4980-88b8-98f0b3b9029f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5111d51d8264a7980ee0dd70d1b5282'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:15.556837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9e2e3862-b25b-4980-88b8-98f0b3b9029f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5111d51d8264a7980ee0dd70d1b5282'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:15.556837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9e2e3862-b25b-4980-88b8-98f0b3b9029f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5111d51d8264a7980ee0dd70d1b5282'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:15.577119 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9e2e3862-b25b-4980-88b8-98f0b3b9029f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5111d51d8264a7980ee0dd70d1b5282'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:15.592404 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 964/1938] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:14 2022] DELETE /identity/v3/users/b5111d51d8264a7980ee0dd70d1b5282 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:15.843494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed7463-b665-446e-a3ca-37ede497a4c6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:15.852052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a5ed7463-b665-446e-a3ca-37ede497a4c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:15.861907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed7463-b665-446e-a3ca-37ede497a4c6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:15.938222 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea53c1-d606-4c78-9570-4b3ddf8675d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:15.939168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea53c1-d606-4c78-9570-4b3ddf8675d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:15.939682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea53c1-d606-4c78-9570-4b3ddf8675d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:15.940262 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea53c1-d606-4c78-9570-4b3ddf8675d8 admin admin] PATH_INFO: `/v3/users/0bfe79a180454ed4984fb8aa20fef175` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:15.942073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea53c1-d606-4c78-9570-4b3ddf8675d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bfe79a180454ed4984fb8aa20fef175)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:15.945830 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea53c1-d606-4c78-9570-4b3ddf8675d8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:16.006973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:16.187396 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e3862-b25b-4980-88b8-98f0b3b9029f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:16.187396 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9e2e3862-b25b-4980-88b8-98f0b3b9029f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:16.202976 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2e3862-b25b-4980-88b8-98f0b3b9029f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:16.225574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddfc708-73fc-4c03-99f8-fe8d47019c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:16.226507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfc708-73fc-4c03-99f8-fe8d47019c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:16.227250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfc708-73fc-4c03-99f8-fe8d47019c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:16.227492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddfc708-73fc-4c03-99f8-fe8d47019c2d admin admin] PATH_INFO: `/v3/projects/4a460a328246402a93a8e724e545632a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:16.228945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfc708-73fc-4c03-99f8-fe8d47019c2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a460a328246402a93a8e724e545632a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:16.232147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddfc708-73fc-4c03-99f8-fe8d47019c2d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:16.394202 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5eea53c1-d606-4c78-9570-4b3ddf8675d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bfe79a180454ed4984fb8aa20fef175'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:16.415588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ddfc708-73fc-4c03-99f8-fe8d47019c2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a460a328246402a93a8e724e545632a', 'request_id': 'req-1ddfc708-73fc-4c03-99f8-fe8d47019c2d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:16.425467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ddfc708-73fc-4c03-99f8-fe8d47019c2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a460a328246402a93a8e724e545632a', 'request_id': 'req-1ddfc708-73fc-4c03-99f8-fe8d47019c2d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:16.586396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5eea53c1-d606-4c78-9570-4b3ddf8675d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bfe79a180454ed4984fb8aa20fef175'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:16.588729 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5eea53c1-d606-4c78-9570-4b3ddf8675d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bfe79a180454ed4984fb8aa20fef175'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:16.589679 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5eea53c1-d606-4c78-9570-4b3ddf8675d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bfe79a180454ed4984fb8aa20fef175'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:16.599204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5eea53c1-d606-4c78-9570-4b3ddf8675d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bfe79a180454ed4984fb8aa20fef175'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:16.622269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 975/1939] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:15 2022] DELETE /identity/v3/users/0bfe79a180454ed4984fb8aa20fef175 => generated 0 bytes in 784 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:16.627066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8165a-04a6-4c14-a983-af05ea6a1462 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:16.628836 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8165a-04a6-4c14-a983-af05ea6a1462 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:16.628836 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8165a-04a6-4c14-a983-af05ea6a1462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:16.652770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7fe8165a-04a6-4c14-a983-af05ea6a1462 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:16.654499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7fe8165a-04a6-4c14-a983-af05ea6a1462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:16.725743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ddfc708-73fc-4c03-99f8-fe8d47019c2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a460a328246402a93a8e724e545632a', 'request_id': 'req-1ddfc708-73fc-4c03-99f8-fe8d47019c2d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:16.726732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 976/1940] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:16.736854 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8165a-04a6-4c14-a983-af05ea6a1462 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:16.738426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7fe8165a-04a6-4c14-a983-af05ea6a1462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:16.793513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ddfc708-73fc-4c03-99f8-fe8d47019c2d admin admin] The token cache is being invalidate because project 4a460a328246402a93a8e724e545632a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:16.793513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ddfc708-73fc-4c03-99f8-fe8d47019c2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:16.800896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1ddfc708-73fc-4c03-99f8-fe8d47019c2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:16.800896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 965/1941] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:16 2022] DELETE /identity/v3/projects/4a460a328246402a93a8e724e545632a => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:16.803457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9671f27-57e0-423f-9770-9332aa06323d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:16.803611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9671f27-57e0-423f-9770-9332aa06323d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:16.803908 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9671f27-57e0-423f-9770-9332aa06323d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:16.875536 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c9671f27-57e0-423f-9770-9332aa06323d None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:16.877201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c9671f27-57e0-423f-9770-9332aa06323d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:16.882651 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8165a-04a6-4c14-a983-af05ea6a1462 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:16.892795 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 966/1942] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:16.904020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff44c0-eadd-4f77-8b09-9ae51c34ecca None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:16.904020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff44c0-eadd-4f77-8b09-9ae51c34ecca None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:16.904949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f4bc5-939c-41a2-8d8d-8e9ee216a2e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:16.914644 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f4bc5-939c-41a2-8d8d-8e9ee216a2e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:16.914644 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f4bc5-939c-41a2-8d8d-8e9ee216a2e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:16.914644 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7f4bc5-939c-41a2-8d8d-8e9ee216a2e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:16.915120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daff44c0-eadd-4f77-8b09-9ae51c34ecca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:16.920602 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ec7f4bc5-939c-41a2-8d8d-8e9ee216a2e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:16.934997 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-daff44c0-eadd-4f77-8b09-9ae51c34ecca None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:16.937547 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-daff44c0-eadd-4f77-8b09-9ae51c34ecca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:16.938965 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f4bc5-939c-41a2-8d8d-8e9ee216a2e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:16.940879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7f4bc5-939c-41a2-8d8d-8e9ee216a2e0 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:16.949764 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 967/1943] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:16.954369 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff44c0-eadd-4f77-8b09-9ae51c34ecca None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:16.956408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 977/1944] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:16.966645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daff44c0-eadd-4f77-8b09-9ae51c34ecca None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:16.974368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fce6b-9be9-4467-bc75-8f3ae08bd541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:16.976147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fce6b-9be9-4467-bc75-8f3ae08bd541 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:16.976758 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fce6b-9be9-4467-bc75-8f3ae08bd541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:16.977450 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757fce6b-9be9-4467-bc75-8f3ae08bd541 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:16.979367 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fce6b-9be9-4467-bc75-8f3ae08bd541 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:16.980429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fce6b-9be9-4467-bc75-8f3ae08bd541 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:16.982443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757fce6b-9be9-4467-bc75-8f3ae08bd541 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:16.998495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 968/1945] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:04:17.003522 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f4bc5-939c-41a2-8d8d-8e9ee216a2e0 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:17.024455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7f4bc5-939c-41a2-8d8d-8e9ee216a2e0 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:17.033657 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e228e2-3d92-4e88-aae2-ab0ab5038e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:17.034291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e228e2-3d92-4e88-aae2-ab0ab5038e22 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:17.034844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e228e2-3d92-4e88-aae2-ab0ab5038e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:17.035279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e228e2-3d92-4e88-aae2-ab0ab5038e22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:17.036142 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e228e2-3d92-4e88-aae2-ab0ab5038e22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:17.038483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e228e2-3d92-4e88-aae2-ab0ab5038e22 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:17.224386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 978/1946] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:17 2022] POST /identity/v3/projects => generated 381 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:17.228695 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fce6b-9be9-4467-bc75-8f3ae08bd541 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:17.249093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757fce6b-9be9-4467-bc75-8f3ae08bd541 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:17.255822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e228e2-3d92-4e88-aae2-ab0ab5038e22 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:17.266885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-18e228e2-3d92-4e88-aae2-ab0ab5038e22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:17.466500 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e228e2-3d92-4e88-aae2-ab0ab5038e22 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:17.485554 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffef6a-14c7-49cf-b5e1-c7e1839dba03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:17.488987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffef6a-14c7-49cf-b5e1-c7e1839dba03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:17.488987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffef6a-14c7-49cf-b5e1-c7e1839dba03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:17.488987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faffef6a-14c7-49cf-b5e1-c7e1839dba03 admin admin] PATH_INFO: `/v3/users/52567607814b4dce8b35cda56df6ad58` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:17.488987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffef6a-14c7-49cf-b5e1-c7e1839dba03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52567607814b4dce8b35cda56df6ad58)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:17.494753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ad252-02e1-4b81-92ba-8e7160dee714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:17.496704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ad252-02e1-4b81-92ba-8e7160dee714 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:17.497142 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ad252-02e1-4b81-92ba-8e7160dee714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:17.497561 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46ad252-02e1-4b81-92ba-8e7160dee714 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:17.500695 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ad252-02e1-4b81-92ba-8e7160dee714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:17.503712 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46ad252-02e1-4b81-92ba-8e7160dee714 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:17.530885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faffef6a-14c7-49cf-b5e1-c7e1839dba03 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:17.644410 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:17.745093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 969/1947] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:17 2022] POST /identity/v3/users => generated 291 bytes in 517 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:17.748476 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ad252-02e1-4b81-92ba-8e7160dee714 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:17.751957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b46ad252-02e1-4b81-92ba-8e7160dee714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:17.768091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46ad252-02e1-4b81-92ba-8e7160dee714 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:17.783100 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf828a4-41fe-46d9-8f68-4b9b5d7af84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:17.783868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf828a4-41fe-46d9-8f68-4b9b5d7af84b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:17.785220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf828a4-41fe-46d9-8f68-4b9b5d7af84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:17.785220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf828a4-41fe-46d9-8f68-4b9b5d7af84b admin admin] PATH_INFO: `/v3/projects/4dec59e1057c4a6d80118b3116ba5276` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:17.785751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf828a4-41fe-46d9-8f68-4b9b5d7af84b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dec59e1057c4a6d80118b3116ba5276)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:17.789166 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf828a4-41fe-46d9-8f68-4b9b5d7af84b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:17.958811 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-faffef6a-14c7-49cf-b5e1-c7e1839dba03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52567607814b4dce8b35cda56df6ad58'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:17.964954 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9bf828a4-41fe-46d9-8f68-4b9b5d7af84b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dec59e1057c4a6d80118b3116ba5276', 'request_id': 'req-9bf828a4-41fe-46d9-8f68-4b9b5d7af84b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:17.974417 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9bf828a4-41fe-46d9-8f68-4b9b5d7af84b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dec59e1057c4a6d80118b3116ba5276', 'request_id': 'req-9bf828a4-41fe-46d9-8f68-4b9b5d7af84b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:18.128562 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-faffef6a-14c7-49cf-b5e1-c7e1839dba03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52567607814b4dce8b35cda56df6ad58'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:18.129430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-faffef6a-14c7-49cf-b5e1-c7e1839dba03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52567607814b4dce8b35cda56df6ad58'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:18.131177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-faffef6a-14c7-49cf-b5e1-c7e1839dba03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52567607814b4dce8b35cda56df6ad58'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:18.142821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-faffef6a-14c7-49cf-b5e1-c7e1839dba03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52567607814b4dce8b35cda56df6ad58'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:18.181639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 979/1948] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:17 2022] DELETE /identity/v3/users/52567607814b4dce8b35cda56df6ad58 => generated 0 bytes in 923 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:18.181639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffef6a-14c7-49cf-b5e1-c7e1839dba03 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:18.182536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-faffef6a-14c7-49cf-b5e1-c7e1839dba03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:18.238544 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9bf828a4-41fe-46d9-8f68-4b9b5d7af84b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dec59e1057c4a6d80118b3116ba5276', 'request_id': 'req-9bf828a4-41fe-46d9-8f68-4b9b5d7af84b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:18.264714 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9bf828a4-41fe-46d9-8f68-4b9b5d7af84b admin admin] The token cache is being invalidate because project 4dec59e1057c4a6d80118b3116ba5276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:18.265351 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9bf828a4-41fe-46d9-8f68-4b9b5d7af84b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:18.266493 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9bf828a4-41fe-46d9-8f68-4b9b5d7af84b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:18.269997 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 970/1949] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:17 2022] DELETE /identity/v3/projects/4dec59e1057c4a6d80118b3116ba5276 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:18.272583 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcca9e-475e-4cba-b39c-d4aa60e9c45c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:18.274363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcca9e-475e-4cba-b39c-d4aa60e9c45c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:18.275354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbcca9e-475e-4cba-b39c-d4aa60e9c45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:18.279621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faffef6a-14c7-49cf-b5e1-c7e1839dba03 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:18.284934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b695507-09ac-46bd-9639-87953c65dd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:18.285656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b695507-09ac-46bd-9639-87953c65dd5f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:18.286600 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b695507-09ac-46bd-9639-87953c65dd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:18.287335 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b695507-09ac-46bd-9639-87953c65dd5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:18.288121 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b695507-09ac-46bd-9639-87953c65dd5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:18.289903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b695507-09ac-46bd-9639-87953c65dd5f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:18.300537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1fbcca9e-475e-4cba-b39c-d4aa60e9c45c None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:18.304771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 980/1950] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:18.304952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1fbcca9e-475e-4cba-b39c-d4aa60e9c45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:18.305037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b695507-09ac-46bd-9639-87953c65dd5f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:18.307105 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3b695507-09ac-46bd-9639-87953c65dd5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:18.314085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 971/1951] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:18.315286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcca9e-475e-4cba-b39c-d4aa60e9c45c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:18.316804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1fbcca9e-475e-4cba-b39c-d4aa60e9c45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:18.326854 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b695507-09ac-46bd-9639-87953c65dd5f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:18.339141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea53564-8c24-4afe-88aa-37b41e2a9b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:18.340505 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea53564-8c24-4afe-88aa-37b41e2a9b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:18.340996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea53564-8c24-4afe-88aa-37b41e2a9b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:18.341135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbcca9e-475e-4cba-b39c-d4aa60e9c45c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:18.341486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea53564-8c24-4afe-88aa-37b41e2a9b11 admin admin] PATH_INFO: `/v3/users/de99b26d12bb4bc9b8b1191663fe3220` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:18.343253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea53564-8c24-4afe-88aa-37b41e2a9b11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de99b26d12bb4bc9b8b1191663fe3220)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:18.346338 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea53564-8c24-4afe-88aa-37b41e2a9b11 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:18.351602 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bbb48-24d1-474b-bac6-746832aaee09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:18.353414 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bbb48-24d1-474b-bac6-746832aaee09 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:18.354201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bbb48-24d1-474b-bac6-746832aaee09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:18.354650 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4bbb48-24d1-474b-bac6-746832aaee09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:18.355775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bbb48-24d1-474b-bac6-746832aaee09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:18.357635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4bbb48-24d1-474b-bac6-746832aaee09 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:18.377491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 972/1952] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:18.379301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bbb48-24d1-474b-bac6-746832aaee09 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:18.382492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9e4bbb48-24d1-474b-bac6-746832aaee09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:18.454651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4bbb48-24d1-474b-bac6-746832aaee09 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:18.460558 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f90af2-4654-4c21-a373-bb67366dcbd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:18.461410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f90af2-4654-4c21-a373-bb67366dcbd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:18.461797 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f90af2-4654-4c21-a373-bb67366dcbd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:18.462185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f90af2-4654-4c21-a373-bb67366dcbd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:18.472298 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f90af2-4654-4c21-a373-bb67366dcbd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:18.473822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f90af2-4654-4c21-a373-bb67366dcbd1 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:18.483985 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 973/1953] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:18.484789 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f90af2-4654-4c21-a373-bb67366dcbd1 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:18.501753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f90af2-4654-4c21-a373-bb67366dcbd1 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:18.501753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7708068-f081-4ba4-8b25-a1490032f189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:18.501753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7708068-f081-4ba4-8b25-a1490032f189 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:18.501753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7708068-f081-4ba4-8b25-a1490032f189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:18.502766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7708068-f081-4ba4-8b25-a1490032f189 admin admin] PATH_INFO: `/v3/projects/4b72f8d86a3c4250b19463eb68fc6d70/users/728f5c48b43a46ceabd2aa4eca027233/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:18.520777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7708068-f081-4ba4-8b25-a1490032f189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b72f8d86a3c4250b19463eb68fc6d70, user_id=728f5c48b43a46ceabd2aa4eca027233, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:18.524204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7708068-f081-4ba4-8b25-a1490032f189 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:18.761369 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 974/1954] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:18 2022] PUT /identity/v3/projects/4b72f8d86a3c4250b19463eb68fc6d70/users/728f5c48b43a46ceabd2aa4eca027233/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:18.767484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cda65f-97ec-4907-a283-394c1c90922f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:18.767811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cda65f-97ec-4907-a283-394c1c90922f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:18.776048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cda65f-97ec-4907-a283-394c1c90922f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:18.821460 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fea53564-8c24-4afe-88aa-37b41e2a9b11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de99b26d12bb4bc9b8b1191663fe3220'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:18.837085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c5cda65f-97ec-4907-a283-394c1c90922f None None] MFA Rules not processed for user `728f5c48b43a46ceabd2aa4eca027233`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:18.838361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c5cda65f-97ec-4907-a283-394c1c90922f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:18.975953 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fea53564-8c24-4afe-88aa-37b41e2a9b11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de99b26d12bb4bc9b8b1191663fe3220'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:18.977062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fea53564-8c24-4afe-88aa-37b41e2a9b11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de99b26d12bb4bc9b8b1191663fe3220'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:18.978847 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fea53564-8c24-4afe-88aa-37b41e2a9b11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de99b26d12bb4bc9b8b1191663fe3220'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:18.981497 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 975/1955] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:18.993210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cda65f-97ec-4907-a283-394c1c90922f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:19.002081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fea53564-8c24-4afe-88aa-37b41e2a9b11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de99b26d12bb4bc9b8b1191663fe3220'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:19.069566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 981/1956] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:18 2022] DELETE /identity/v3/users/de99b26d12bb4bc9b8b1191663fe3220 => generated 0 bytes in 767 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:19.113207 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:19.121231 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cda65f-97ec-4907-a283-394c1c90922f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:19.231624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974724a-8063-480d-bcaa-872f02403478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:19.232308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974724a-8063-480d-bcaa-872f02403478 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:19.232595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974724a-8063-480d-bcaa-872f02403478 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:19.232885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7974724a-8063-480d-bcaa-872f02403478 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:19.233541 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974724a-8063-480d-bcaa-872f02403478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:19.235616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7974724a-8063-480d-bcaa-872f02403478 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:19.310208 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea53564-8c24-4afe-88aa-37b41e2a9b11 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:19.312228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fea53564-8c24-4afe-88aa-37b41e2a9b11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:19.335337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea53564-8c24-4afe-88aa-37b41e2a9b11 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:19.343015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d839df-83a9-4c43-97bf-5b6bca00b50f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:19.343696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d839df-83a9-4c43-97bf-5b6bca00b50f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:19.343986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d839df-83a9-4c43-97bf-5b6bca00b50f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:19.344309 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d839df-83a9-4c43-97bf-5b6bca00b50f admin admin] PATH_INFO: `/v3/projects/2d51270a9feb460599c2b76b3774546b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:19.345651 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d839df-83a9-4c43-97bf-5b6bca00b50f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d51270a9feb460599c2b76b3774546b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:19.350779 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d839df-83a9-4c43-97bf-5b6bca00b50f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:19.398442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 976/1957] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:18 2022] POST /identity/v3/projects => generated 381 bytes in 416 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:19.401773 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974724a-8063-480d-bcaa-872f02403478 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:19.403596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7974724a-8063-480d-bcaa-872f02403478 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:19.495863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7974724a-8063-480d-bcaa-872f02403478 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:19.501168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6334f05-2f6c-4568-a0cf-8e97041a1145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:19.502971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6334f05-2f6c-4568-a0cf-8e97041a1145 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:19.502971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6334f05-2f6c-4568-a0cf-8e97041a1145 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:19.503637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6334f05-2f6c-4568-a0cf-8e97041a1145 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:19.503840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6334f05-2f6c-4568-a0cf-8e97041a1145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:19.506955 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6334f05-2f6c-4568-a0cf-8e97041a1145 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:19.646922 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71d839df-83a9-4c43-97bf-5b6bca00b50f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d51270a9feb460599c2b76b3774546b', 'request_id': 'req-71d839df-83a9-4c43-97bf-5b6bca00b50f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:19.651364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 977/1958] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:19 2022] POST /identity/v3/users => generated 297 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:19.653790 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6334f05-2f6c-4568-a0cf-8e97041a1145 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:19.659099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71d839df-83a9-4c43-97bf-5b6bca00b50f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d51270a9feb460599c2b76b3774546b', 'request_id': 'req-71d839df-83a9-4c43-97bf-5b6bca00b50f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:19.667817 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6334f05-2f6c-4568-a0cf-8e97041a1145 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:19.777556 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ddcd4-914a-4048-bc61-dd28f8b358f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:19.778373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ddcd4-914a-4048-bc61-dd28f8b358f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:19.778644 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ddcd4-914a-4048-bc61-dd28f8b358f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:19.778919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ddcd4-914a-4048-bc61-dd28f8b358f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:19.780584 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ddcd4-914a-4048-bc61-dd28f8b358f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:19.782199 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ddcd4-914a-4048-bc61-dd28f8b358f4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:19.799584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71d839df-83a9-4c43-97bf-5b6bca00b50f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d51270a9feb460599c2b76b3774546b', 'request_id': 'req-71d839df-83a9-4c43-97bf-5b6bca00b50f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:19.801299 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 978/1959] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:19.809144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ddcd4-914a-4048-bc61-dd28f8b358f4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:19.816870 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71d839df-83a9-4c43-97bf-5b6bca00b50f admin admin] The token cache is being invalidate because project 2d51270a9feb460599c2b76b3774546b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:19.817322 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71d839df-83a9-4c43-97bf-5b6bca00b50f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:19.818306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71d839df-83a9-4c43-97bf-5b6bca00b50f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:19.820293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 982/1960] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:19 2022] DELETE /identity/v3/projects/2d51270a9feb460599c2b76b3774546b => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:19.824315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d839df-83a9-4c43-97bf-5b6bca00b50f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:19.825844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ddcd4-914a-4048-bc61-dd28f8b358f4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:19.826736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-71d839df-83a9-4c43-97bf-5b6bca00b50f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:19.837215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e63a8c-c25a-4035-9cd2-1d4023f0593a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:19.838153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e63a8c-c25a-4035-9cd2-1d4023f0593a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:19.838601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e63a8c-c25a-4035-9cd2-1d4023f0593a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:19.838601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e63a8c-c25a-4035-9cd2-1d4023f0593a admin admin] PATH_INFO: `/v3/projects/a199ea10fd4d4a3c8b1ee19750d19620/users/f23749b7fe684f25908781f1b83c6b66/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:19.844438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d839df-83a9-4c43-97bf-5b6bca00b50f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:19.852150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fd27f-35ca-4681-bb40-4b051dbd92f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:19.853434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fd27f-35ca-4681-bb40-4b051dbd92f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:19.854029 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fd27f-35ca-4681-bb40-4b051dbd92f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:19.854499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6fd27f-35ca-4681-bb40-4b051dbd92f1 admin admin] PATH_INFO: `/v3/users/e307c03e8057433ab2be0773897c375b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:19.856378 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fd27f-35ca-4681-bb40-4b051dbd92f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e307c03e8057433ab2be0773897c375b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:19.859568 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e63a8c-c25a-4035-9cd2-1d4023f0593a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a199ea10fd4d4a3c8b1ee19750d19620, user_id=f23749b7fe684f25908781f1b83c6b66, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:19.860002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6fd27f-35ca-4681-bb40-4b051dbd92f1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:19.862435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e63a8c-c25a-4035-9cd2-1d4023f0593a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:20.090518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 979/1961] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:19 2022] PUT /identity/v3/projects/a199ea10fd4d4a3c8b1ee19750d19620/users/f23749b7fe684f25908781f1b83c6b66/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:20.091683 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e63a8c-c25a-4035-9cd2-1d4023f0593a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:20.095292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b2e63a8c-c25a-4035-9cd2-1d4023f0593a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:20.171607 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e63a8c-c25a-4035-9cd2-1d4023f0593a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:20.177463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb9a4be-a015-40be-ad47-8b7d271e9018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:20.178111 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9a4be-a015-40be-ad47-8b7d271e9018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:20.178392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9a4be-a015-40be-ad47-8b7d271e9018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:20.178668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb9a4be-a015-40be-ad47-8b7d271e9018 admin admin] PATH_INFO: `/v3/projects/5b81eed50e0d4b0bb05a5b00c6be1818` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:20.179945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9a4be-a015-40be-ad47-8b7d271e9018 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b81eed50e0d4b0bb05a5b00c6be1818)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:20.181883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb9a4be-a015-40be-ad47-8b7d271e9018 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:20.273735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8a6fd27f-35ca-4681-bb40-4b051dbd92f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e307c03e8057433ab2be0773897c375b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:20.383012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8a6fd27f-35ca-4681-bb40-4b051dbd92f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e307c03e8057433ab2be0773897c375b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:20.384058 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8a6fd27f-35ca-4681-bb40-4b051dbd92f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e307c03e8057433ab2be0773897c375b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:20.387386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8a6fd27f-35ca-4681-bb40-4b051dbd92f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e307c03e8057433ab2be0773897c375b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:20.410603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8a6fd27f-35ca-4681-bb40-4b051dbd92f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e307c03e8057433ab2be0773897c375b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:20.434135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1eb9a4be-a015-40be-ad47-8b7d271e9018 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b81eed50e0d4b0bb05a5b00c6be1818', 'request_id': 'req-1eb9a4be-a015-40be-ad47-8b7d271e9018'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:20.456147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1eb9a4be-a015-40be-ad47-8b7d271e9018 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b81eed50e0d4b0bb05a5b00c6be1818', 'request_id': 'req-1eb9a4be-a015-40be-ad47-8b7d271e9018'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:20.465768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 983/1962] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:19 2022] DELETE /identity/v3/users/e307c03e8057433ab2be0773897c375b => generated 0 bytes in 642 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:20.467213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fd27f-35ca-4681-bb40-4b051dbd92f1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:20.469292 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8a6fd27f-35ca-4681-bb40-4b051dbd92f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:20.603976 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1eb9a4be-a015-40be-ad47-8b7d271e9018 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b81eed50e0d4b0bb05a5b00c6be1818', 'request_id': 'req-1eb9a4be-a015-40be-ad47-8b7d271e9018'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:20.633566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1eb9a4be-a015-40be-ad47-8b7d271e9018 admin admin] The token cache is being invalidate because project 5b81eed50e0d4b0bb05a5b00c6be1818 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:20.634384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1eb9a4be-a015-40be-ad47-8b7d271e9018 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:20.635961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6fd27f-35ca-4681-bb40-4b051dbd92f1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:20.636342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1eb9a4be-a015-40be-ad47-8b7d271e9018 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:20.639277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 980/1963] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:20 2022] DELETE /identity/v3/projects/5b81eed50e0d4b0bb05a5b00c6be1818 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:20.644983 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95432bc-e9bd-4850-a254-5baf74cf0857 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:20.647340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d83b3-8a5e-463d-ad42-cb625593811c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:20.648167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95432bc-e9bd-4850-a254-5baf74cf0857 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:20.648348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d83b3-8a5e-463d-ad42-cb625593811c admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:20.648962 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d83b3-8a5e-463d-ad42-cb625593811c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:20.649414 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7d83b3-8a5e-463d-ad42-cb625593811c admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:20.649681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95432bc-e9bd-4850-a254-5baf74cf0857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:20.650645 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d83b3-8a5e-463d-ad42-cb625593811c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:20.653368 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7d83b3-8a5e-463d-ad42-cb625593811c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:20.666384 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 984/1964] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:20 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:20.673793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d83b3-8a5e-463d-ad42-cb625593811c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:20.676932 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ed7d83b3-8a5e-463d-ad42-cb625593811c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:20.698884 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7d83b3-8a5e-463d-ad42-cb625593811c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:20.703702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d95432bc-e9bd-4850-a254-5baf74cf0857 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:20.703702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d95432bc-e9bd-4850-a254-5baf74cf0857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:20.707221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ff3e5-fab5-43c5-8ffa-b9cdf0639545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:20.708479 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ff3e5-fab5-43c5-8ffa-b9cdf0639545 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:20.709064 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ff3e5-fab5-43c5-8ffa-b9cdf0639545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:20.709543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ff3e5-fab5-43c5-8ffa-b9cdf0639545 admin admin] PATH_INFO: `/v3/projects/a199ea10fd4d4a3c8b1ee19750d19620/users/f23749b7fe684f25908781f1b83c6b66/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:20.711603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ff3e5-fab5-43c5-8ffa-b9cdf0639545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a199ea10fd4d4a3c8b1ee19750d19620, user_id=f23749b7fe684f25908781f1b83c6b66, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:20.712399 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 981/1965] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:20.714367 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678ff3e5-fab5-43c5-8ffa-b9cdf0639545 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:20.725236 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49311dd-70e7-4cb2-aeeb-8350889be242 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:20.725526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49311dd-70e7-4cb2-aeeb-8350889be242 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:20.725796 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49311dd-70e7-4cb2-aeeb-8350889be242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:20.730912 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 985/1966] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:20 2022] PUT /identity/v3/projects/a199ea10fd4d4a3c8b1ee19750d19620/users/f23749b7fe684f25908781f1b83c6b66/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:20.741521 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f05525-d966-4599-be53-635eba17afed None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:20.741851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f05525-d966-4599-be53-635eba17afed None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:20.742072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f05525-d966-4599-be53-635eba17afed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:20.755366 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b49311dd-70e7-4cb2-aeeb-8350889be242 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:20.759434 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b49311dd-70e7-4cb2-aeeb-8350889be242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:20.775027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-86f05525-d966-4599-be53-635eba17afed None None] MFA Rules not processed for user `f23749b7fe684f25908781f1b83c6b66`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:20.776836 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-86f05525-d966-4599-be53-635eba17afed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:20.818957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:20.967913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 986/1967] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:20 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:21.004125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 982/1968] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:21.014239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f05525-d966-4599-be53-635eba17afed None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:21.017300 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49311dd-70e7-4cb2-aeeb-8350889be242 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:21.021430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b49311dd-70e7-4cb2-aeeb-8350889be242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:21.040902 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f05525-d966-4599-be53-635eba17afed None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:21.057748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46722689-a552-4950-8e16-ea032bc0eac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:21.059549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46722689-a552-4950-8e16-ea032bc0eac4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:21.063830 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46722689-a552-4950-8e16-ea032bc0eac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:21.064356 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46722689-a552-4950-8e16-ea032bc0eac4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:21.065521 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46722689-a552-4950-8e16-ea032bc0eac4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:21.066417 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46722689-a552-4950-8e16-ea032bc0eac4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:21.072635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46722689-a552-4950-8e16-ea032bc0eac4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:21.099358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 987/1969] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:04:21.100648 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46722689-a552-4950-8e16-ea032bc0eac4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:21.119305 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46722689-a552-4950-8e16-ea032bc0eac4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:21.127097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ef02b-b246-4815-830a-523545ec3262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:21.127794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ef02b-b246-4815-830a-523545ec3262 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:21.128083 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ef02b-b246-4815-830a-523545ec3262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:21.128389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0ef02b-b246-4815-830a-523545ec3262 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:21.129317 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ef02b-b246-4815-830a-523545ec3262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:21.132344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0ef02b-b246-4815-830a-523545ec3262 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:21.215594 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49311dd-70e7-4cb2-aeeb-8350889be242 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:21.224475 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63afccb-37b2-4b30-9537-e4802fa006dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:21.225672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63afccb-37b2-4b30-9537-e4802fa006dc service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:21.226147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63afccb-37b2-4b30-9537-e4802fa006dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:21.227286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63afccb-37b2-4b30-9537-e4802fa006dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:21.245812 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63afccb-37b2-4b30-9537-e4802fa006dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:21.248903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63afccb-37b2-4b30-9537-e4802fa006dc service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:21.271537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 983/1970] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:21 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:21.303437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 988/1971] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:21 2022] POST /identity/v3/projects => generated 379 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:21.305778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63afccb-37b2-4b30-9537-e4802fa006dc service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:21.330637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63afccb-37b2-4b30-9537-e4802fa006dc service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:21.442948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271176d-4243-49f0-adc9-fc6ef9e0a9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:21.443432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271176d-4243-49f0-adc9-fc6ef9e0a9ed admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:21.443716 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271176d-4243-49f0-adc9-fc6ef9e0a9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:21.443989 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f271176d-4243-49f0-adc9-fc6ef9e0a9ed admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:21.444636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271176d-4243-49f0-adc9-fc6ef9e0a9ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:21.446821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f271176d-4243-49f0-adc9-fc6ef9e0a9ed admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:21.606207 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 984/1972] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:21 2022] POST /identity/v3/users => generated 290 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:21.608597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ef02b-b246-4815-830a-523545ec3262 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:21.623455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0ef02b-b246-4815-830a-523545ec3262 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:21.632766 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4154b33-237d-4c37-b7e4-86aa959a1368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:21.634394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4154b33-237d-4c37-b7e4-86aa959a1368 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:21.635232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4154b33-237d-4c37-b7e4-86aa959a1368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:21.635774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4154b33-237d-4c37-b7e4-86aa959a1368 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:21.636752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4154b33-237d-4c37-b7e4-86aa959a1368 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:21.639688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4154b33-237d-4c37-b7e4-86aa959a1368 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:21.655790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 989/1973] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:21.662902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271176d-4243-49f0-adc9-fc6ef9e0a9ed admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:21.687387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f271176d-4243-49f0-adc9-fc6ef9e0a9ed admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:21.724829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a032d-9067-4a86-99b1-f51e6b130520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:21.725542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a032d-9067-4a86-99b1-f51e6b130520 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:21.725829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a032d-9067-4a86-99b1-f51e6b130520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:21.726110 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54a032d-9067-4a86-99b1-f51e6b130520 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:21.727288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a032d-9067-4a86-99b1-f51e6b130520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:21.730111 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54a032d-9067-4a86-99b1-f51e6b130520 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:21.763069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4154b33-237d-4c37-b7e4-86aa959a1368 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:21.764000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 985/1974] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:21 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:21.769150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a4154b33-237d-4c37-b7e4-86aa959a1368 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:21.770718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a032d-9067-4a86-99b1-f51e6b130520 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:21.804872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4154b33-237d-4c37-b7e4-86aa959a1368 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:21.818571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89997e8-1cf4-4909-a530-d2f7843beeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:21.818571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89997e8-1cf4-4909-a530-d2f7843beeae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:21.818571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89997e8-1cf4-4909-a530-d2f7843beeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:21.820670 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89997e8-1cf4-4909-a530-d2f7843beeae admin admin] PATH_INFO: `/v3/projects/b29973e19d714c77b5c1da229b056cca` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:21.823352 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89997e8-1cf4-4909-a530-d2f7843beeae admin admin] RBAC: Authorizing `identity:delete_project(project_id=b29973e19d714c77b5c1da229b056cca)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:21.877773 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54a032d-9067-4a86-99b1-f51e6b130520 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:21.880869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89997e8-1cf4-4909-a530-d2f7843beeae admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:21.902411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c030a4-bc8b-4aaf-812d-fe9d71b6497f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:21.905128 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c030a4-bc8b-4aaf-812d-fe9d71b6497f admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:21.906721 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c030a4-bc8b-4aaf-812d-fe9d71b6497f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:21.907278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c030a4-bc8b-4aaf-812d-fe9d71b6497f admin admin] PATH_INFO: `/v3/projects/3c77dd3a7d684048a3a98282d0264c09/users/91757786a7e24aa8969cbc8d7bf75073/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:21.943874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c030a4-bc8b-4aaf-812d-fe9d71b6497f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c77dd3a7d684048a3a98282d0264c09, user_id=91757786a7e24aa8969cbc8d7bf75073, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:21.948718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c030a4-bc8b-4aaf-812d-fe9d71b6497f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:22.016748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:22.061069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 986/1975] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:21 2022] PUT /identity/v3/projects/3c77dd3a7d684048a3a98282d0264c09/users/91757786a7e24aa8969cbc8d7bf75073/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:22.083632 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c3108-d321-42bf-97c5-67525f2cd95b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:22.084253 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c3108-d321-42bf-97c5-67525f2cd95b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:22.084656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c3108-d321-42bf-97c5-67525f2cd95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:22.130935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e89997e8-1cf4-4909-a530-d2f7843beeae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b29973e19d714c77b5c1da229b056cca', 'request_id': 'req-e89997e8-1cf4-4909-a530-d2f7843beeae'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:22.146192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-679c3108-d321-42bf-97c5-67525f2cd95b None None] MFA Rules not processed for user `91757786a7e24aa8969cbc8d7bf75073`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:22.146192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-679c3108-d321-42bf-97c5-67525f2cd95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:22.152023 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e89997e8-1cf4-4909-a530-d2f7843beeae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b29973e19d714c77b5c1da229b056cca', 'request_id': 'req-e89997e8-1cf4-4909-a530-d2f7843beeae'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:22.262538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 987/1976] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:22.269243 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c3108-d321-42bf-97c5-67525f2cd95b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:22.282948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e89997e8-1cf4-4909-a530-d2f7843beeae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b29973e19d714c77b5c1da229b056cca', 'request_id': 'req-e89997e8-1cf4-4909-a530-d2f7843beeae'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:22.325430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e89997e8-1cf4-4909-a530-d2f7843beeae admin admin] The token cache is being invalidate because project b29973e19d714c77b5c1da229b056cca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:22.335313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e89997e8-1cf4-4909-a530-d2f7843beeae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:22.336485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e89997e8-1cf4-4909-a530-d2f7843beeae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:22.342774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 990/1977] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:21 2022] DELETE /identity/v3/projects/b29973e19d714c77b5c1da229b056cca => generated 0 bytes in 576 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:22.345158 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff65eea-ddf6-4c61-b120-31ac358f9a21 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:22.345646 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff65eea-ddf6-4c61-b120-31ac358f9a21 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:22.346075 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff65eea-ddf6-4c61-b120-31ac358f9a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:22.389538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4ff65eea-ddf6-4c61-b120-31ac358f9a21 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:22.391600 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4ff65eea-ddf6-4c61-b120-31ac358f9a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:22.432764 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c3108-d321-42bf-97c5-67525f2cd95b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:22.440639 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e798c4-4784-4d59-8239-6bb405ba821b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:22.441383 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e798c4-4784-4d59-8239-6bb405ba821b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:22.441667 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e798c4-4784-4d59-8239-6bb405ba821b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:22.442108 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e798c4-4784-4d59-8239-6bb405ba821b admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:22.443149 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e798c4-4784-4d59-8239-6bb405ba821b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:22.445405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e798c4-4784-4d59-8239-6bb405ba821b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:22.498746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 991/1978] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:22.508756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bfd72-4561-4b91-a8b9-79dcb6d4f096 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:22.509024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bfd72-4561-4b91-a8b9-79dcb6d4f096 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:22.509304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443bfd72-4561-4b91-a8b9-79dcb6d4f096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:22.539254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-443bfd72-4561-4b91-a8b9-79dcb6d4f096 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:22.540565 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-443bfd72-4561-4b91-a8b9-79dcb6d4f096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:22.553003 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 992/1979] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:22.558873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bfd72-4561-4b91-a8b9-79dcb6d4f096 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:22.570762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443bfd72-4561-4b91-a8b9-79dcb6d4f096 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:22.576310 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c172db3-6e4b-429f-a2c2-c9ba0fee448b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:22.576945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c172db3-6e4b-429f-a2c2-c9ba0fee448b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:22.577229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c172db3-6e4b-429f-a2c2-c9ba0fee448b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:22.577504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c172db3-6e4b-429f-a2c2-c9ba0fee448b admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:22.578214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c172db3-6e4b-429f-a2c2-c9ba0fee448b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:22.578520 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c172db3-6e4b-429f-a2c2-c9ba0fee448b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:22.580176 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c172db3-6e4b-429f-a2c2-c9ba0fee448b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:22.593514 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 993/1980] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:04:22.597814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c172db3-6e4b-429f-a2c2-c9ba0fee448b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:22.608719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c172db3-6e4b-429f-a2c2-c9ba0fee448b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:22.614320 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2112f03-8c14-4b05-9b19-de577584242f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:22.614987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2112f03-8c14-4b05-9b19-de577584242f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:22.615332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2112f03-8c14-4b05-9b19-de577584242f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:22.615760 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2112f03-8c14-4b05-9b19-de577584242f admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:22.616534 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2112f03-8c14-4b05-9b19-de577584242f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:22.618523 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2112f03-8c14-4b05-9b19-de577584242f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:22.673067 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 988/1981] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:22 2022] POST /identity/v3/projects => generated 379 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:22.677802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e798c4-4784-4d59-8239-6bb405ba821b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:22.679614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-19e798c4-4784-4d59-8239-6bb405ba821b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:22.767149 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e798c4-4784-4d59-8239-6bb405ba821b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:22.775196 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 994/1982] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:22 2022] POST /identity/v3/projects => generated 367 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:22.776842 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260a3ff-d613-4408-80ee-971fd449a4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:22.777725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260a3ff-d613-4408-80ee-971fd449a4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:22.778588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260a3ff-d613-4408-80ee-971fd449a4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:22.779060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d260a3ff-d613-4408-80ee-971fd449a4e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:22.779896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2112f03-8c14-4b05-9b19-de577584242f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:22.780802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260a3ff-d613-4408-80ee-971fd449a4e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:22.784780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d260a3ff-d613-4408-80ee-971fd449a4e0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:22.895893 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2112f03-8c14-4b05-9b19-de577584242f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:22.907920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c9df4-3b3c-4366-ae19-56d5fcaa5a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:22.908601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c9df4-3b3c-4366-ae19-56d5fcaa5a16 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:22.908886 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c9df4-3b3c-4366-ae19-56d5fcaa5a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:22.909153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c9df4-3b3c-4366-ae19-56d5fcaa5a16 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:22.909973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c9df4-3b3c-4366-ae19-56d5fcaa5a16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:22.912252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9c9df4-3b3c-4366-ae19-56d5fcaa5a16 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:22.973170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 989/1983] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:22 2022] POST /identity/v3/users => generated 290 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:22.973992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260a3ff-d613-4408-80ee-971fd449a4e0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:22.978338 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d260a3ff-d613-4408-80ee-971fd449a4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:22.999256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d260a3ff-d613-4408-80ee-971fd449a4e0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:23.005581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdfc8f0-efbc-45f3-9b97-fa2e796895ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:23.006306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfc8f0-efbc-45f3-9b97-fa2e796895ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:23.006591 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfc8f0-efbc-45f3-9b97-fa2e796895ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:23.006862 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdfc8f0-efbc-45f3-9b97-fa2e796895ac admin admin] PATH_INFO: `/v3/users/728f5c48b43a46ceabd2aa4eca027233` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:23.008122 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdfc8f0-efbc-45f3-9b97-fa2e796895ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=728f5c48b43a46ceabd2aa4eca027233)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:23.010314 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdfc8f0-efbc-45f3-9b97-fa2e796895ac admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:23.158454 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:23.164381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 995/1984] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:22 2022] POST /identity/v3/users => generated 284 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:23.165650 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c9df4-3b3c-4366-ae19-56d5fcaa5a16 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:23.178210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c9df4-3b3c-4366-ae19-56d5fcaa5a16 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:23.183030 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88400baf-cffb-4ef2-b8d9-d4b3f19e6bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:23.183694 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88400baf-cffb-4ef2-b8d9-d4b3f19e6bac admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:23.184494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88400baf-cffb-4ef2-b8d9-d4b3f19e6bac admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:23.184494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88400baf-cffb-4ef2-b8d9-d4b3f19e6bac admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:23.184912 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88400baf-cffb-4ef2-b8d9-d4b3f19e6bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:23.186549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88400baf-cffb-4ef2-b8d9-d4b3f19e6bac admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:23.199508 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 996/1985] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:23.200834 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88400baf-cffb-4ef2-b8d9-d4b3f19e6bac admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:23.225098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88400baf-cffb-4ef2-b8d9-d4b3f19e6bac admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:23.225098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf49ac1-95ac-4716-8362-b6fbea13e23e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:23.225098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf49ac1-95ac-4716-8362-b6fbea13e23e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:23.225098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf49ac1-95ac-4716-8362-b6fbea13e23e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:23.225456 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf49ac1-95ac-4716-8362-b6fbea13e23e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:23.225456 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf49ac1-95ac-4716-8362-b6fbea13e23e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:23.225456 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf49ac1-95ac-4716-8362-b6fbea13e23e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:23.237763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 997/1986] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:23.237763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf49ac1-95ac-4716-8362-b6fbea13e23e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:23.247957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf49ac1-95ac-4716-8362-b6fbea13e23e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:23.252635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109c307-d87a-4a86-85c6-749e6e303d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:23.253364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109c307-d87a-4a86-85c6-749e6e303d6d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:23.253639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109c307-d87a-4a86-85c6-749e6e303d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:23.253915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6109c307-d87a-4a86-85c6-749e6e303d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:23.260121 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109c307-d87a-4a86-85c6-749e6e303d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:23.260121 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6109c307-d87a-4a86-85c6-749e6e303d6d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:23.268260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 998/1987] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:23.269098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109c307-d87a-4a86-85c6-749e6e303d6d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:23.278993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6109c307-d87a-4a86-85c6-749e6e303d6d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:23.283671 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fa6fa-3377-4b0a-8290-4cda97de6ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:23.284264 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fa6fa-3377-4b0a-8290-4cda97de6ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:23.284547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fa6fa-3377-4b0a-8290-4cda97de6ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:23.284816 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008fa6fa-3377-4b0a-8290-4cda97de6ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:23.285439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fa6fa-3377-4b0a-8290-4cda97de6ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:23.287667 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008fa6fa-3377-4b0a-8290-4cda97de6ec9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:23.303192 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 999/1988] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:23.303192 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fa6fa-3377-4b0a-8290-4cda97de6ec9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:23.313938 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008fa6fa-3377-4b0a-8290-4cda97de6ec9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:23.322132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a25f5-3c5d-497b-a0be-3b5fd55b7ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:23.323047 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a25f5-3c5d-497b-a0be-3b5fd55b7ad9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:23.323535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a25f5-3c5d-497b-a0be-3b5fd55b7ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:23.324082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a25f5-3c5d-497b-a0be-3b5fd55b7ad9 admin admin] PATH_INFO: `/v3/projects/0ea87c171af840a0ace0242344273207/users/5cabc1ac02484d87a2f2318bd6524104/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:23.345656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a25f5-3c5d-497b-a0be-3b5fd55b7ad9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ea87c171af840a0ace0242344273207, user_id=5cabc1ac02484d87a2f2318bd6524104, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:23.351812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a25f5-3c5d-497b-a0be-3b5fd55b7ad9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:23.373126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-acdfc8f0-efbc-45f3-9b97-fa2e796895ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '728f5c48b43a46ceabd2aa4eca027233'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:23.483036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-acdfc8f0-efbc-45f3-9b97-fa2e796895ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '728f5c48b43a46ceabd2aa4eca027233'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:23.484062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-acdfc8f0-efbc-45f3-9b97-fa2e796895ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '728f5c48b43a46ceabd2aa4eca027233'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:23.484983 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-acdfc8f0-efbc-45f3-9b97-fa2e796895ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '728f5c48b43a46ceabd2aa4eca027233'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:23.498735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1000/1989] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:23 2022] PUT /identity/v3/projects/0ea87c171af840a0ace0242344273207/users/5cabc1ac02484d87a2f2318bd6524104/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:23.501751 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a25f5-3c5d-497b-a0be-3b5fd55b7ad9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:23.505713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c70a25f5-3c5d-497b-a0be-3b5fd55b7ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:23.516054 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-acdfc8f0-efbc-45f3-9b97-fa2e796895ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '728f5c48b43a46ceabd2aa4eca027233'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:23.531516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 990/1990] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:22 2022] DELETE /identity/v3/users/728f5c48b43a46ceabd2aa4eca027233 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:23.533244 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29c9bb-76f7-418d-a58c-d8d86f880621 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:23.533870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29c9bb-76f7-418d-a58c-d8d86f880621 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:23.534157 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29c9bb-76f7-418d-a58c-d8d86f880621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:23.574656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-da29c9bb-76f7-418d-a58c-d8d86f880621 None None] MFA Rules not processed for user `5cabc1ac02484d87a2f2318bd6524104`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:23.575971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-da29c9bb-76f7-418d-a58c-d8d86f880621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:23.636027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a25f5-3c5d-497b-a0be-3b5fd55b7ad9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:23.665657 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 991/1991] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:23.667035 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729a0fe-f340-4642-bc34-cac12be47247 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:23.667325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729a0fe-f340-4642-bc34-cac12be47247 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:23.667601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a729a0fe-f340-4642-bc34-cac12be47247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:23.700618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a729a0fe-f340-4642-bc34-cac12be47247 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:23.702543 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a729a0fe-f340-4642-bc34-cac12be47247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:23.711730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3d525-3257-4376-9206-d5cb44e47c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:23.713094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3d525-3257-4376-9206-d5cb44e47c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:23.713483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3d525-3257-4376-9206-d5cb44e47c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:23.713877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d3d525-3257-4376-9206-d5cb44e47c17 admin admin] PATH_INFO: `/v3/projects/f9bbf8fedb084b7eb76d336b80f97aa1/users/b2a9a2e58d2c431daee612486d3814c4/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:23.716188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 992/1992] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:23.717171 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729a0fe-f340-4642-bc34-cac12be47247 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:23.718723 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a729a0fe-f340-4642-bc34-cac12be47247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:23.733755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a729a0fe-f340-4642-bc34-cac12be47247 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:23.744775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297ed9f7-9770-4c87-87c0-de325810991d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:23.745966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ed9f7-9770-4c87-87c0-de325810991d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:23.746209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3d525-3257-4376-9206-d5cb44e47c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9bbf8fedb084b7eb76d336b80f97aa1, user_id=b2a9a2e58d2c431daee612486d3814c4, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:23.746433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ed9f7-9770-4c87-87c0-de325810991d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:23.746876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297ed9f7-9770-4c87-87c0-de325810991d admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:23.747504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ed9f7-9770-4c87-87c0-de325810991d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:23.749738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297ed9f7-9770-4c87-87c0-de325810991d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:23.750752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d3d525-3257-4376-9206-d5cb44e47c17 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:23.870366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1001/1993] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:23 2022] PUT /identity/v3/projects/f9bbf8fedb084b7eb76d336b80f97aa1/users/b2a9a2e58d2c431daee612486d3814c4/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:23.871363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3d525-3257-4376-9206-d5cb44e47c17 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:23.874153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e7d3d525-3257-4376-9206-d5cb44e47c17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:23.925657 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 993/1994] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:23 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:23.925657 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da61c6-c0c5-4447-9a48-4699f94821b1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:23.925657 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da61c6-c0c5-4447-9a48-4699f94821b1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:23.925657 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da61c6-c0c5-4447-9a48-4699f94821b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:23.947132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e5da61c6-c0c5-4447-9a48-4699f94821b1 None None] MFA Rules not processed for user `b2a9a2e58d2c431daee612486d3814c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:23.948500 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e5da61c6-c0c5-4447-9a48-4699f94821b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:23.948736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d3d525-3257-4376-9206-d5cb44e47c17 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:24.037554 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde13b4-033c-40e2-9cc1-fb8e3303b7bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:24.038418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde13b4-033c-40e2-9cc1-fb8e3303b7bf service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:24.038706 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde13b4-033c-40e2-9cc1-fb8e3303b7bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:24.038980 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde13b4-033c-40e2-9cc1-fb8e3303b7bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:24.041411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 994/1995] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:24.042474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da61c6-c0c5-4447-9a48-4699f94821b1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:24.120879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde13b4-033c-40e2-9cc1-fb8e3303b7bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:24.122681 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde13b4-033c-40e2-9cc1-fb8e3303b7bf service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:24.134557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1002/1996] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:24.135776 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde13b4-033c-40e2-9cc1-fb8e3303b7bf service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:24.145094 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da61c6-c0c5-4447-9a48-4699f94821b1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:24.146916 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde13b4-033c-40e2-9cc1-fb8e3303b7bf service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:24.152394 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d6aea-9987-4109-bfc5-fbdf711cdd7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:24.153135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d6aea-9987-4109-bfc5-fbdf711cdd7b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:24.153484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d6aea-9987-4109-bfc5-fbdf711cdd7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:24.153829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d6aea-9987-4109-bfc5-fbdf711cdd7b admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:24.153963 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13551f-910c-4349-b6c1-d7525033d02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:24.154604 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13551f-910c-4349-b6c1-d7525033d02f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:24.154719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d6aea-9987-4109-bfc5-fbdf711cdd7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:24.155339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13551f-910c-4349-b6c1-d7525033d02f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:24.155339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb13551f-910c-4349-b6c1-d7525033d02f admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:24.156759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13551f-910c-4349-b6c1-d7525033d02f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:24.157483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d6aea-9987-4109-bfc5-fbdf711cdd7b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:24.159235 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb13551f-910c-4349-b6c1-d7525033d02f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:24.285863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 995/1997] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:24 2022] POST /identity/v3/users => generated 295 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:24.291217 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d6aea-9987-4109-bfc5-fbdf711cdd7b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:24.314245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d6aea-9987-4109-bfc5-fbdf711cdd7b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:24.323107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d68787-5a95-49f3-a2b7-2b0b527f1ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:24.323774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d68787-5a95-49f3-a2b7-2b0b527f1ff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:24.324085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d68787-5a95-49f3-a2b7-2b0b527f1ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:24.324362 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d68787-5a95-49f3-a2b7-2b0b527f1ff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:24.325015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d68787-5a95-49f3-a2b7-2b0b527f1ff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:24.326653 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d68787-5a95-49f3-a2b7-2b0b527f1ff3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:24.362189 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 996/1998] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:24.367752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1003/1999] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:24 2022] POST /identity/v3/projects => generated 367 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:24.369800 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d68787-5a95-49f3-a2b7-2b0b527f1ff3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:24.370429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13551f-910c-4349-b6c1-d7525033d02f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:24.387185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d68787-5a95-49f3-a2b7-2b0b527f1ff3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:24.491487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb13551f-910c-4349-b6c1-d7525033d02f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:24.514572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2b162-ee80-4dc0-9ec9-5035b3b38acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:24.516387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2b162-ee80-4dc0-9ec9-5035b3b38acc admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:24.517499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2b162-ee80-4dc0-9ec9-5035b3b38acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:24.518370 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca2b162-ee80-4dc0-9ec9-5035b3b38acc admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:24.521061 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2b162-ee80-4dc0-9ec9-5035b3b38acc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:24.526701 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca2b162-ee80-4dc0-9ec9-5035b3b38acc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:24.569145 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324dc44-e305-4194-b86c-55bbb8acdfca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:24.570497 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324dc44-e305-4194-b86c-55bbb8acdfca admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:24.571048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324dc44-e305-4194-b86c-55bbb8acdfca admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:24.571588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324dc44-e305-4194-b86c-55bbb8acdfca admin admin] PATH_INFO: `/v3/projects/9848f41b49f141b885411aba8ccfe54e/users/616f284d65684086bcf448ef06061c66/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:24.597382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324dc44-e305-4194-b86c-55bbb8acdfca admin admin] RBAC: Authorizing `identity:create_grant(project_id=9848f41b49f141b885411aba8ccfe54e, user_id=616f284d65684086bcf448ef06061c66, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:24.601400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324dc44-e305-4194-b86c-55bbb8acdfca admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:24.769434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1004/2000] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:24 2022] POST /identity/v3/users => generated 290 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:24.778436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2b162-ee80-4dc0-9ec9-5035b3b38acc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:24.778787 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 997/2001] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:24 2022] PUT /identity/v3/projects/9848f41b49f141b885411aba8ccfe54e/users/616f284d65684086bcf448ef06061c66/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:24.783482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324dc44-e305-4194-b86c-55bbb8acdfca admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:24.787327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca2b162-ee80-4dc0-9ec9-5035b3b38acc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:24.882278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:24.882883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324dc44-e305-4194-b86c-55bbb8acdfca admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:24.886657 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373f2e4-c968-435b-8303-f85c8e94dae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:24.887381 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373f2e4-c968-435b-8303-f85c8e94dae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:24.887693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373f2e4-c968-435b-8303-f85c8e94dae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:24.887957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373f2e4-c968-435b-8303-f85c8e94dae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:24.888613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373f2e4-c968-435b-8303-f85c8e94dae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:24.890330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373f2e4-c968-435b-8303-f85c8e94dae8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:24.897109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089cec9-17d7-42a8-a4dc-8f301248fd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:24.898761 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089cec9-17d7-42a8-a4dc-8f301248fd3b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:24.898761 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089cec9-17d7-42a8-a4dc-8f301248fd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:24.898761 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089cec9-17d7-42a8-a4dc-8f301248fd3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:24.906005 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089cec9-17d7-42a8-a4dc-8f301248fd3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:24.912226 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 998/2002] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:24.912226 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373f2e4-c968-435b-8303-f85c8e94dae8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:24.912226 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f373f2e4-c968-435b-8303-f85c8e94dae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:24.914801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3089cec9-17d7-42a8-a4dc-8f301248fd3b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:24.932605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1005/2003] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:24 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:24.934155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089cec9-17d7-42a8-a4dc-8f301248fd3b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:24.936134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373f2e4-c968-435b-8303-f85c8e94dae8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:24.942115 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ac10f-e84d-466c-903d-33637eb6112d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:24.943341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ac10f-e84d-466c-903d-33637eb6112d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:24.944779 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ac10f-e84d-466c-903d-33637eb6112d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:24.945831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ac10f-e84d-466c-903d-33637eb6112d admin admin] PATH_INFO: `/v3/projects/4b72f8d86a3c4250b19463eb68fc6d70` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:24.947211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ac10f-e84d-466c-903d-33637eb6112d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b72f8d86a3c4250b19463eb68fc6d70)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:24.948801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089cec9-17d7-42a8-a4dc-8f301248fd3b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:24.949168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4ac10f-e84d-466c-903d-33637eb6112d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:24.958747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda271be-8508-4c9b-b1b5-c4e207618cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:24.959440 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda271be-8508-4c9b-b1b5-c4e207618cf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:24.959756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda271be-8508-4c9b-b1b5-c4e207618cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:24.960532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda271be-8508-4c9b-b1b5-c4e207618cf1 admin admin] PATH_INFO: `/v3/projects/9848f41b49f141b885411aba8ccfe54e/users/616f284d65684086bcf448ef06061c66/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:24.963477 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda271be-8508-4c9b-b1b5-c4e207618cf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9848f41b49f141b885411aba8ccfe54e, user_id=616f284d65684086bcf448ef06061c66, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:24.973129 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda271be-8508-4c9b-b1b5-c4e207618cf1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:25.034536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1006/2004] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:24 2022] PUT /identity/v3/projects/9848f41b49f141b885411aba8ccfe54e/users/616f284d65684086bcf448ef06061c66/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:25.045522 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda271be-8508-4c9b-b1b5-c4e207618cf1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:25.153807 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda271be-8508-4c9b-b1b5-c4e207618cf1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:25.162963 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233b3115-2341-4efa-96fd-37efeafe06a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:25.164077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b3115-2341-4efa-96fd-37efeafe06a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:25.164655 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b3115-2341-4efa-96fd-37efeafe06a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:25.165227 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233b3115-2341-4efa-96fd-37efeafe06a9 admin admin] PATH_INFO: `/v3/projects/2d2c585eeef746d6b92d855a6a91be18/users/ca70ddc2924f4b7ba763c54da544b11e/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:25.200987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b3115-2341-4efa-96fd-37efeafe06a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d2c585eeef746d6b92d855a6a91be18, user_id=ca70ddc2924f4b7ba763c54da544b11e, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:25.203818 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233b3115-2341-4efa-96fd-37efeafe06a9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:25.212876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3a4ac10f-e84d-466c-903d-33637eb6112d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b72f8d86a3c4250b19463eb68fc6d70', 'request_id': 'req-3a4ac10f-e84d-466c-903d-33637eb6112d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:25.220143 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3a4ac10f-e84d-466c-903d-33637eb6112d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b72f8d86a3c4250b19463eb68fc6d70', 'request_id': 'req-3a4ac10f-e84d-466c-903d-33637eb6112d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:25.343357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1007/2005] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:25 2022] PUT /identity/v3/projects/2d2c585eeef746d6b92d855a6a91be18/users/ca70ddc2924f4b7ba763c54da544b11e/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:25.349882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c0416-2d6c-4779-8847-33bba2917e34 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:25.350277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c0416-2d6c-4779-8847-33bba2917e34 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:25.350525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4c0416-2d6c-4779-8847-33bba2917e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:25.376507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3a4ac10f-e84d-466c-903d-33637eb6112d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b72f8d86a3c4250b19463eb68fc6d70', 'request_id': 'req-3a4ac10f-e84d-466c-903d-33637eb6112d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:25.392164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6b4c0416-2d6c-4779-8847-33bba2917e34 None None] MFA Rules not processed for user `616f284d65684086bcf448ef06061c66`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:25.392164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6b4c0416-2d6c-4779-8847-33bba2917e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:25.397915 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3a4ac10f-e84d-466c-903d-33637eb6112d admin admin] The token cache is being invalidate because project 4b72f8d86a3c4250b19463eb68fc6d70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:25.398275 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3a4ac10f-e84d-466c-903d-33637eb6112d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:25.399215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3a4ac10f-e84d-466c-903d-33637eb6112d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:25.405553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 999/2006] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:24 2022] DELETE /identity/v3/projects/4b72f8d86a3c4250b19463eb68fc6d70 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:25.405553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ac10f-e84d-466c-903d-33637eb6112d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:25.405553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3a4ac10f-e84d-466c-903d-33637eb6112d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:25.499818 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1008/2007] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:25.504825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c0416-2d6c-4779-8847-33bba2917e34 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:25.506323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6b4c0416-2d6c-4779-8847-33bba2917e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:25.521747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ac10f-e84d-466c-903d-33637eb6112d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:25.626354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4c0416-2d6c-4779-8847-33bba2917e34 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:25.635035 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbd606-e251-4d03-bd15-5a07b6afc290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:25.635256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd445982-8205-42eb-8f0d-40739d93f276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:25.635388 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbd606-e251-4d03-bd15-5a07b6afc290 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:25.635468 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd445982-8205-42eb-8f0d-40739d93f276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:25.635468 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd445982-8205-42eb-8f0d-40739d93f276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:25.635594 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbd606-e251-4d03-bd15-5a07b6afc290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:25.635683 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd445982-8205-42eb-8f0d-40739d93f276 admin admin] PATH_INFO: `/v3/users/f23749b7fe684f25908781f1b83c6b66` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:25.635757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dbd606-e251-4d03-bd15-5a07b6afc290 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:25.635757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbd606-e251-4d03-bd15-5a07b6afc290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:25.636969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd445982-8205-42eb-8f0d-40739d93f276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f23749b7fe684f25908781f1b83c6b66)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:25.637087 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dbd606-e251-4d03-bd15-5a07b6afc290 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:25.640363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd445982-8205-42eb-8f0d-40739d93f276 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:25.651240 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1000/2008] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:25 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:25.652432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbd606-e251-4d03-bd15-5a07b6afc290 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:25.654340 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-99dbd606-e251-4d03-bd15-5a07b6afc290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:25.735414 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dbd606-e251-4d03-bd15-5a07b6afc290 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:25.739354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fd7d0-636d-4c94-aff5-584cde5a41b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:25.740116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fd7d0-636d-4c94-aff5-584cde5a41b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:25.740400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fd7d0-636d-4c94-aff5-584cde5a41b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:25.740672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fd7d0-636d-4c94-aff5-584cde5a41b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:25.751519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fd7d0-636d-4c94-aff5-584cde5a41b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:25.756030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fd7d0-636d-4c94-aff5-584cde5a41b6 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:25.768190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1001/2009] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:25 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:25.771233 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fd7d0-636d-4c94-aff5-584cde5a41b6 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:25.786326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fd7d0-636d-4c94-aff5-584cde5a41b6 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:25.792140 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581d922-e092-4d51-9919-ee56ddfc3ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:25.792825 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581d922-e092-4d51-9919-ee56ddfc3ee4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:25.793174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581d922-e092-4d51-9919-ee56ddfc3ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:25.793584 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581d922-e092-4d51-9919-ee56ddfc3ee4 admin admin] PATH_INFO: `/v3/projects/2d2c585eeef746d6b92d855a6a91be18/users/ca70ddc2924f4b7ba763c54da544b11e/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:25.795926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9581d922-e092-4d51-9919-ee56ddfc3ee4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d2c585eeef746d6b92d855a6a91be18, user_id=ca70ddc2924f4b7ba763c54da544b11e, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:25.798549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9581d922-e092-4d51-9919-ee56ddfc3ee4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:25.823699 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1002/2010] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:25 2022] PUT /identity/v3/projects/2d2c585eeef746d6b92d855a6a91be18/users/ca70ddc2924f4b7ba763c54da544b11e/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:25.833634 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d79c21-00fc-4a8f-a867-b0832e117b30 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:25.834107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d79c21-00fc-4a8f-a867-b0832e117b30 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:25.834577 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d79c21-00fc-4a8f-a867-b0832e117b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:25.882117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-95d79c21-00fc-4a8f-a867-b0832e117b30 None None] MFA Rules not processed for user `ca70ddc2924f4b7ba763c54da544b11e`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:25.882117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-95d79c21-00fc-4a8f-a867-b0832e117b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:25.983027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd445982-8205-42eb-8f0d-40739d93f276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f23749b7fe684f25908781f1b83c6b66'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:26.046095 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1003/2011] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:25 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:26.047430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d79c21-00fc-4a8f-a867-b0832e117b30 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:26.138509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd445982-8205-42eb-8f0d-40739d93f276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f23749b7fe684f25908781f1b83c6b66'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:26.139725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd445982-8205-42eb-8f0d-40739d93f276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f23749b7fe684f25908781f1b83c6b66'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:26.140191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd445982-8205-42eb-8f0d-40739d93f276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f23749b7fe684f25908781f1b83c6b66'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:26.152104 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dd445982-8205-42eb-8f0d-40739d93f276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f23749b7fe684f25908781f1b83c6b66'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:26.189777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d79c21-00fc-4a8f-a867-b0832e117b30 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:26.202515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1009/2012] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:25 2022] DELETE /identity/v3/users/f23749b7fe684f25908781f1b83c6b66 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:26.204152 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e069ba8-6b8a-489a-ae51-4f4c8ab0897a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:26.204995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e069ba8-6b8a-489a-ae51-4f4c8ab0897a service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:26.205352 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e069ba8-6b8a-489a-ae51-4f4c8ab0897a service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:26.205950 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e069ba8-6b8a-489a-ae51-4f4c8ab0897a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:26.207351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd445982-8205-42eb-8f0d-40739d93f276 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:26.209097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4e069ba8-6b8a-489a-ae51-4f4c8ab0897a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:26.210473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dd445982-8205-42eb-8f0d-40739d93f276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:26.361616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e069ba8-6b8a-489a-ae51-4f4c8ab0897a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:26.363428 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd445982-8205-42eb-8f0d-40739d93f276 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:26.363544 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e069ba8-6b8a-489a-ae51-4f4c8ab0897a service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:26.369457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ecb98-1900-40c5-87c4-60e4b3da2383 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:26.373968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ecb98-1900-40c5-87c4-60e4b3da2383 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:26.374857 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ecb98-1900-40c5-87c4-60e4b3da2383 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:26.375345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ecb98-1900-40c5-87c4-60e4b3da2383 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:26.379289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-429ecb98-1900-40c5-87c4-60e4b3da2383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:26.388596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1004/2013] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:26.516749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ecb98-1900-40c5-87c4-60e4b3da2383 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:26.521231 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ecb98-1900-40c5-87c4-60e4b3da2383 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:26.547331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1010/2014] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:26.747015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e069ba8-6b8a-489a-ae51-4f4c8ab0897a service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:26.759321 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e069ba8-6b8a-489a-ae51-4f4c8ab0897a service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:26.765751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4f5576-b47c-445f-a369-1dd6394902ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:26.766789 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f5576-b47c-445f-a369-1dd6394902ea service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:26.767236 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f5576-b47c-445f-a369-1dd6394902ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:26.767671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4f5576-b47c-445f-a369-1dd6394902ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:26.772448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2c4f5576-b47c-445f-a369-1dd6394902ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:26.890782 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f5576-b47c-445f-a369-1dd6394902ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:26.892348 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4f5576-b47c-445f-a369-1dd6394902ea service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:26.909940 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1005/2015] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:27.065864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ecb98-1900-40c5-87c4-60e4b3da2383 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:27.067482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-429ecb98-1900-40c5-87c4-60e4b3da2383 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:27.176011 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ecb98-1900-40c5-87c4-60e4b3da2383 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:27.181839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d1ed30-2b4d-4709-a366-00a34d4fe18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:27.183106 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1ed30-2b4d-4709-a366-00a34d4fe18c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:27.183106 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1ed30-2b4d-4709-a366-00a34d4fe18c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:27.183106 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d1ed30-2b4d-4709-a366-00a34d4fe18c admin admin] PATH_INFO: `/v3/projects/a199ea10fd4d4a3c8b1ee19750d19620` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:27.183106 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1ed30-2b4d-4709-a366-00a34d4fe18c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a199ea10fd4d4a3c8b1ee19750d19620)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:27.183106 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d1ed30-2b4d-4709-a366-00a34d4fe18c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:27.421620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e3d1ed30-2b4d-4709-a366-00a34d4fe18c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a199ea10fd4d4a3c8b1ee19750d19620', 'request_id': 'req-e3d1ed30-2b4d-4709-a366-00a34d4fe18c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:27.432145 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e3d1ed30-2b4d-4709-a366-00a34d4fe18c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a199ea10fd4d4a3c8b1ee19750d19620', 'request_id': 'req-e3d1ed30-2b4d-4709-a366-00a34d4fe18c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:27.587084 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e3d1ed30-2b4d-4709-a366-00a34d4fe18c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a199ea10fd4d4a3c8b1ee19750d19620', 'request_id': 'req-e3d1ed30-2b4d-4709-a366-00a34d4fe18c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:27.615514 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e3d1ed30-2b4d-4709-a366-00a34d4fe18c admin admin] The token cache is being invalidate because project a199ea10fd4d4a3c8b1ee19750d19620 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:27.615968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e3d1ed30-2b4d-4709-a366-00a34d4fe18c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:27.617177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e3d1ed30-2b4d-4709-a366-00a34d4fe18c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:27.619581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1011/2016] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:27 2022] DELETE /identity/v3/projects/a199ea10fd4d4a3c8b1ee19750d19620 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:27.624495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b06ec-ac11-4762-836e-d0b54a5e1747 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:27.624808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b06ec-ac11-4762-836e-d0b54a5e1747 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:27.625086 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b06ec-ac11-4762-836e-d0b54a5e1747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:27.648203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b20b06ec-ac11-4762-836e-d0b54a5e1747 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:27.649467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b20b06ec-ac11-4762-836e-d0b54a5e1747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:27.767999 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1006/2017] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:27.779200 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b05c06-0fb7-42c9-8f09-927c175a1dc3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:27.780096 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b05c06-0fb7-42c9-8f09-927c175a1dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:27.780764 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b05c06-0fb7-42c9-8f09-927c175a1dc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:27.783118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:27.811218 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-39b05c06-0fb7-42c9-8f09-927c175a1dc3 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:27.812617 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-39b05c06-0fb7-42c9-8f09-927c175a1dc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:27.822791 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1012/2018] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:27.835448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b06ec-ac11-4762-836e-d0b54a5e1747 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:27.846068 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b06ec-ac11-4762-836e-d0b54a5e1747 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:27.853344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16f454-0692-4662-a8b5-7028f0ac2082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:27.854185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16f454-0692-4662-a8b5-7028f0ac2082 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:27.854481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16f454-0692-4662-a8b5-7028f0ac2082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:27.854745 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16f454-0692-4662-a8b5-7028f0ac2082 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:27.855407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16f454-0692-4662-a8b5-7028f0ac2082 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:27.855749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16f454-0692-4662-a8b5-7028f0ac2082 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:27.857410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16f454-0692-4662-a8b5-7028f0ac2082 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:27.892141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1007/2019] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:04:27.906017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b05c06-0fb7-42c9-8f09-927c175a1dc3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:27.919268 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b05c06-0fb7-42c9-8f09-927c175a1dc3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:27.928104 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b9bef-ae2c-4a81-af45-784d9f6aaaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:27.928709 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b9bef-ae2c-4a81-af45-784d9f6aaaad admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:27.929035 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b9bef-ae2c-4a81-af45-784d9f6aaaad admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:27.929311 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b9bef-ae2c-4a81-af45-784d9f6aaaad admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:27.929955 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b9bef-ae2c-4a81-af45-784d9f6aaaad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:27.932591 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9b9bef-ae2c-4a81-af45-784d9f6aaaad admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:28.128052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1013/2020] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:27 2022] POST /identity/v3/projects => generated 377 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:28.134101 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16f454-0692-4662-a8b5-7028f0ac2082 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:28.146153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16f454-0692-4662-a8b5-7028f0ac2082 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:28.275616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ca275-6d6f-4ded-bbc6-91145dade222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:28.281619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ca275-6d6f-4ded-bbc6-91145dade222 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:28.281619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ca275-6d6f-4ded-bbc6-91145dade222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:28.281619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ca275-6d6f-4ded-bbc6-91145dade222 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:28.281619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ca275-6d6f-4ded-bbc6-91145dade222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:28.282729 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ca275-6d6f-4ded-bbc6-91145dade222 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:28.407391 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1008/2021] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:28 2022] POST /identity/v3/users => generated 289 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:28.408236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b9bef-ae2c-4a81-af45-784d9f6aaaad admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:28.428783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b9bef-ae2c-4a81-af45-784d9f6aaaad admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:28.438145 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf45c31-81ab-4f0c-a5dd-e445fa983f52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:28.438780 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf45c31-81ab-4f0c-a5dd-e445fa983f52 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:28.439372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf45c31-81ab-4f0c-a5dd-e445fa983f52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:28.439827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf45c31-81ab-4f0c-a5dd-e445fa983f52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:28.440498 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf45c31-81ab-4f0c-a5dd-e445fa983f52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:28.443267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf45c31-81ab-4f0c-a5dd-e445fa983f52 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:28.464552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ca275-6d6f-4ded-bbc6-91145dade222 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:28.467784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1014/2022] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:04:28.478844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ca275-6d6f-4ded-bbc6-91145dade222 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:28.484442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3779a3c4-6c6e-46cb-bac9-03e7e64b40b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:28.485282 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779a3c4-6c6e-46cb-bac9-03e7e64b40b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:28.485624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779a3c4-6c6e-46cb-bac9-03e7e64b40b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:28.485910 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3779a3c4-6c6e-46cb-bac9-03e7e64b40b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:28.492430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779a3c4-6c6e-46cb-bac9-03e7e64b40b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:28.492430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3779a3c4-6c6e-46cb-bac9-03e7e64b40b6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:28.528848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1009/2023] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:28.534008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf45c31-81ab-4f0c-a5dd-e445fa983f52 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:28.561663 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf45c31-81ab-4f0c-a5dd-e445fa983f52 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:28.571452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d96965-a226-4793-97f5-300ccec3824a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:28.572479 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d96965-a226-4793-97f5-300ccec3824a admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:28.572900 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d96965-a226-4793-97f5-300ccec3824a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:28.573006 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d96965-a226-4793-97f5-300ccec3824a admin admin] PATH_INFO: `/v3/projects/b7dc54fef855410193af3b417a32abc8/users/963df3106f3442c1ab0f141c24157632/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:28.595569 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d96965-a226-4793-97f5-300ccec3824a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7dc54fef855410193af3b417a32abc8, user_id=963df3106f3442c1ab0f141c24157632, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:28.601956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d96965-a226-4793-97f5-300ccec3824a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:28.809568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1015/2024] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:28 2022] PUT /identity/v3/projects/b7dc54fef855410193af3b417a32abc8/users/963df3106f3442c1ab0f141c24157632/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:28.829518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62460e9-4aac-4d74-aef4-35809e30ff48 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:28.829956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62460e9-4aac-4d74-aef4-35809e30ff48 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:28.830788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62460e9-4aac-4d74-aef4-35809e30ff48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:28.903441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a62460e9-4aac-4d74-aef4-35809e30ff48 None None] MFA Rules not processed for user `963df3106f3442c1ab0f141c24157632`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:28.905652 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a62460e9-4aac-4d74-aef4-35809e30ff48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:29.073514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1010/2025] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:29.075785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d96965-a226-4793-97f5-300ccec3824a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:29.252247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d96965-a226-4793-97f5-300ccec3824a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:29.260945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb7e40-f078-4625-a013-413cd0449166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:29.262032 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb7e40-f078-4625-a013-413cd0449166 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:29.262601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb7e40-f078-4625-a013-413cd0449166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:29.263054 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb7e40-f078-4625-a013-413cd0449166 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:29.264128 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb7e40-f078-4625-a013-413cd0449166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:29.268242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb7e40-f078-4625-a013-413cd0449166 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:29.479814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1016/2026] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:29 2022] POST /identity/v3/projects => generated 375 bytes in 405 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:29.482167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62460e9-4aac-4d74-aef4-35809e30ff48 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:29.498529 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62460e9-4aac-4d74-aef4-35809e30ff48 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:29.565202 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:29.600106 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905629a-7642-4bb0-97f3-9cb5d89be687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:29.600849 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905629a-7642-4bb0-97f3-9cb5d89be687 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:29.601135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905629a-7642-4bb0-97f3-9cb5d89be687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:29.601412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905629a-7642-4bb0-97f3-9cb5d89be687 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:29.602088 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905629a-7642-4bb0-97f3-9cb5d89be687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:29.604215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905629a-7642-4bb0-97f3-9cb5d89be687 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:29.806619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1011/2027] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:29 2022] POST /identity/v3/users => generated 288 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:29.806825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb7e40-f078-4625-a013-413cd0449166 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:29.813995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb7e40-f078-4625-a013-413cd0449166 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:29.820443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11d885-f011-448b-b8ae-cde7c6e076d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:29.821570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11d885-f011-448b-b8ae-cde7c6e076d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:29.822033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11d885-f011-448b-b8ae-cde7c6e076d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:29.822473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11d885-f011-448b-b8ae-cde7c6e076d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:29.823315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11d885-f011-448b-b8ae-cde7c6e076d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:29.825002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11d885-f011-448b-b8ae-cde7c6e076d6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:29.837950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1017/2028] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:29.843612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905629a-7642-4bb0-97f3-9cb5d89be687 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:29.870999 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905629a-7642-4bb0-97f3-9cb5d89be687 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:29.879029 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4e1aea-6195-44b8-8f3d-bd6d89c6288d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:29.880788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e1aea-6195-44b8-8f3d-bd6d89c6288d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:29.880788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e1aea-6195-44b8-8f3d-bd6d89c6288d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:29.880788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4e1aea-6195-44b8-8f3d-bd6d89c6288d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:29.880788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e1aea-6195-44b8-8f3d-bd6d89c6288d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:29.882668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4e1aea-6195-44b8-8f3d-bd6d89c6288d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:29.901474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1012/2029] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:29.907420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11d885-f011-448b-b8ae-cde7c6e076d6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:29.919578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11d885-f011-448b-b8ae-cde7c6e076d6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:29.925301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e363c2-7054-4b59-b1f8-2f7445477797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:29.926106 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e363c2-7054-4b59-b1f8-2f7445477797 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:29.926656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e363c2-7054-4b59-b1f8-2f7445477797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:29.926656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e363c2-7054-4b59-b1f8-2f7445477797 admin admin] PATH_INFO: `/v3/projects/3adca5d2175f4856b8551614813e6612/users/6780f39a47e244c0824f880e88d3a235/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:29.945788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e363c2-7054-4b59-b1f8-2f7445477797 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3adca5d2175f4856b8551614813e6612, user_id=6780f39a47e244c0824f880e88d3a235, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:29.948846 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e363c2-7054-4b59-b1f8-2f7445477797 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:30.048853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1018/2030] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:29 2022] PUT /identity/v3/projects/3adca5d2175f4856b8551614813e6612/users/6780f39a47e244c0824f880e88d3a235/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:30.060281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0633b3-96b5-49d7-8519-70cd75ec45d5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:30.060988 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0633b3-96b5-49d7-8519-70cd75ec45d5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:30.060988 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0633b3-96b5-49d7-8519-70cd75ec45d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:30.097252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ef0633b3-96b5-49d7-8519-70cd75ec45d5 None None] MFA Rules not processed for user `6780f39a47e244c0824f880e88d3a235`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:30.097252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ef0633b3-96b5-49d7-8519-70cd75ec45d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:30.188673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1013/2031] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:30.190808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e363c2-7054-4b59-b1f8-2f7445477797 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:30.266509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e363c2-7054-4b59-b1f8-2f7445477797 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:30.272536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8221ad5-6970-41bc-9e65-bbbced384a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:30.273154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8221ad5-6970-41bc-9e65-bbbced384a34 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:30.273430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8221ad5-6970-41bc-9e65-bbbced384a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:30.273701 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8221ad5-6970-41bc-9e65-bbbced384a34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:30.274376 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8221ad5-6970-41bc-9e65-bbbced384a34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:30.276342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8221ad5-6970-41bc-9e65-bbbced384a34 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:30.506253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1019/2032] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:30 2022] POST /identity/v3/projects => generated 375 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:30.514689 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0633b3-96b5-49d7-8519-70cd75ec45d5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:30.531868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0633b3-96b5-49d7-8519-70cd75ec45d5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:30.616930 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7d094-d21f-4d24-8339-1c924d28fefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:30.617445 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7d094-d21f-4d24-8339-1c924d28fefb admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:30.617781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7d094-d21f-4d24-8339-1c924d28fefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:30.618063 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f7d094-d21f-4d24-8339-1c924d28fefb admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:30.618706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7d094-d21f-4d24-8339-1c924d28fefb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:30.620723 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f7d094-d21f-4d24-8339-1c924d28fefb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:30.779818 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1014/2033] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:30 2022] POST /identity/v3/users => generated 294 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:30.785107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8221ad5-6970-41bc-9e65-bbbced384a34 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:30.802949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8221ad5-6970-41bc-9e65-bbbced384a34 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:30.810944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b2878-81c5-49de-b559-9c3fd435281d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:30.812059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b2878-81c5-49de-b559-9c3fd435281d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:30.812539 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b2878-81c5-49de-b559-9c3fd435281d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:30.812965 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b2878-81c5-49de-b559-9c3fd435281d admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:30.813985 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b2878-81c5-49de-b559-9c3fd435281d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:30.815640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b2878-81c5-49de-b559-9c3fd435281d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:30.829918 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1020/2034] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:30.840491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7d094-d21f-4d24-8339-1c924d28fefb admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:30.852784 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f7d094-d21f-4d24-8339-1c924d28fefb admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:30.863208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844227a-a824-4bec-8326-f2fd2ca1c071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:30.864155 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844227a-a824-4bec-8326-f2fd2ca1c071 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:30.864445 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844227a-a824-4bec-8326-f2fd2ca1c071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:30.864729 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7844227a-a824-4bec-8326-f2fd2ca1c071 admin admin] PATH_INFO: `/v3/projects/d044ffb98bf94c55994753650de92de5/users/0eb1ba23d4004a20b7220c8471a9382c/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:30.896971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844227a-a824-4bec-8326-f2fd2ca1c071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d044ffb98bf94c55994753650de92de5, user_id=0eb1ba23d4004a20b7220c8471a9382c, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:30.902223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7844227a-a824-4bec-8326-f2fd2ca1c071 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:31.062221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1015/2035] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:30 2022] PUT /identity/v3/projects/d044ffb98bf94c55994753650de92de5/users/0eb1ba23d4004a20b7220c8471a9382c/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:31.068485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b2878-81c5-49de-b559-9c3fd435281d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:31.078491 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b2878-81c5-49de-b559-9c3fd435281d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:31.147943 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4efb06-97d3-434e-b1f7-0adcadd0e98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:31.148476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4efb06-97d3-434e-b1f7-0adcadd0e98a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:31.148758 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4efb06-97d3-434e-b1f7-0adcadd0e98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:31.149033 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4efb06-97d3-434e-b1f7-0adcadd0e98a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:31.149684 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4efb06-97d3-434e-b1f7-0adcadd0e98a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:31.151286 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4efb06-97d3-434e-b1f7-0adcadd0e98a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:31.162669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1021/2036] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:31.170990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844227a-a824-4bec-8326-f2fd2ca1c071 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:31.184864 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7844227a-a824-4bec-8326-f2fd2ca1c071 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:31.195634 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dd231-a9bf-4243-91f6-1c476e6eef05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:31.196384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dd231-a9bf-4243-91f6-1c476e6eef05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:31.196666 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dd231-a9bf-4243-91f6-1c476e6eef05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:31.196971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42dd231-a9bf-4243-91f6-1c476e6eef05 admin admin] PATH_INFO: `/v3/projects/d044ffb98bf94c55994753650de92de5/users/0eb1ba23d4004a20b7220c8471a9382c/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:31.199078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42dd231-a9bf-4243-91f6-1c476e6eef05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d044ffb98bf94c55994753650de92de5, user_id=0eb1ba23d4004a20b7220c8471a9382c, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:31.202110 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42dd231-a9bf-4243-91f6-1c476e6eef05 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:31.216269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1016/2037] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:31 2022] PUT /identity/v3/projects/d044ffb98bf94c55994753650de92de5/users/0eb1ba23d4004a20b7220c8471a9382c/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:31.225476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa55957-22d0-4ccb-bde5-524f7eea09fa None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:31.225765 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa55957-22d0-4ccb-bde5-524f7eea09fa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:31.226048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa55957-22d0-4ccb-bde5-524f7eea09fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:31.275050 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-aaa55957-22d0-4ccb-bde5-524f7eea09fa None None] MFA Rules not processed for user `0eb1ba23d4004a20b7220c8471a9382c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:31.276449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-aaa55957-22d0-4ccb-bde5-524f7eea09fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:31.351540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1022/2038] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:31 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:31.363449 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dd231-a9bf-4243-91f6-1c476e6eef05 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:31.365822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a42dd231-a9bf-4243-91f6-1c476e6eef05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:31.460408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42dd231-a9bf-4243-91f6-1c476e6eef05 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:31.464223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f02d5-bd96-4c47-95d4-da60039a3029 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:31.464875 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f02d5-bd96-4c47-95d4-da60039a3029 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:31.465217 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f02d5-bd96-4c47-95d4-da60039a3029 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:31.465496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f02d5-bd96-4c47-95d4-da60039a3029 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:31.527708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f02d5-bd96-4c47-95d4-da60039a3029 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:31.530204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f02d5-bd96-4c47-95d4-da60039a3029 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:31.542937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1017/2039] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:31.857231 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=1,keystone:SELECT=94,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:32.862190 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:35.154670 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa55957-22d0-4ccb-bde5-524f7eea09fa None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:35.156683 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-aaa55957-22d0-4ccb-bde5-524f7eea09fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:35.187490 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:35.242161 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa55957-22d0-4ccb-bde5-524f7eea09fa None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:35.253556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96504de9-b248-4093-97c6-c2a8cf9e9885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:35.253556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96504de9-b248-4093-97c6-c2a8cf9e9885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:35.253556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96504de9-b248-4093-97c6-c2a8cf9e9885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:35.253556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96504de9-b248-4093-97c6-c2a8cf9e9885 admin admin] PATH_INFO: `/v3/users/b2a9a2e58d2c431daee612486d3814c4` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:35.254046 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96504de9-b248-4093-97c6-c2a8cf9e9885 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2a9a2e58d2c431daee612486d3814c4)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:35.254046 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:35.256068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96504de9-b248-4093-97c6-c2a8cf9e9885 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:35.632084 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-96504de9-b248-4093-97c6-c2a8cf9e9885 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2a9a2e58d2c431daee612486d3814c4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:35.756194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f02d5-bd96-4c47-95d4-da60039a3029 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:35.763231 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-96504de9-b248-4093-97c6-c2a8cf9e9885 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2a9a2e58d2c431daee612486d3814c4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:35.764930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-96504de9-b248-4093-97c6-c2a8cf9e9885 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2a9a2e58d2c431daee612486d3814c4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:35.765665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-96504de9-b248-4093-97c6-c2a8cf9e9885 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2a9a2e58d2c431daee612486d3814c4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:35.767343 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f02d5-bd96-4c47-95d4-da60039a3029 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:35.771464 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa467e4-c2b2-417d-8070-1b13b0ab0e2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:35.772159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa467e4-c2b2-417d-8070-1b13b0ab0e2f service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:35.772586 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa467e4-c2b2-417d-8070-1b13b0ab0e2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:35.772866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa467e4-c2b2-417d-8070-1b13b0ab0e2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:35.776803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-96504de9-b248-4093-97c6-c2a8cf9e9885 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2a9a2e58d2c431daee612486d3814c4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:35.776994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dfa467e4-c2b2-417d-8070-1b13b0ab0e2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:35.794051 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1023/2040] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:35 2022] DELETE /identity/v3/users/b2a9a2e58d2c431daee612486d3814c4 => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:35.802008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b148c9-ad63-4f22-82ac-3208e26f9edf None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:35.802440 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b148c9-ad63-4f22-82ac-3208e26f9edf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:35.802917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b148c9-ad63-4f22-82ac-3208e26f9edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:35.825484 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f4b148c9-ad63-4f22-82ac-3208e26f9edf None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:35.826954 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f4b148c9-ad63-4f22-82ac-3208e26f9edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:35.841761 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa467e4-c2b2-417d-8070-1b13b0ab0e2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:35.843339 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa467e4-c2b2-417d-8070-1b13b0ab0e2f service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:35.852991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1018/2041] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:35 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:35.895396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1024/2042] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:35.906122 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa467e4-c2b2-417d-8070-1b13b0ab0e2f service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:35.907635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dfa467e4-c2b2-417d-8070-1b13b0ab0e2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:35.971835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa467e4-c2b2-417d-8070-1b13b0ab0e2f service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:36.034317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c42c5-2ec4-4f20-a904-ceea79f29f71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:36.037318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c42c5-2ec4-4f20-a904-ceea79f29f71 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:36.037318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c42c5-2ec4-4f20-a904-ceea79f29f71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:36.037318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190c42c5-2ec4-4f20-a904-ceea79f29f71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:36.050801 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c42c5-2ec4-4f20-a904-ceea79f29f71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:36.050801 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190c42c5-2ec4-4f20-a904-ceea79f29f71 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:36.059058 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1019/2043] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:36.421611 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b148c9-ad63-4f22-82ac-3208e26f9edf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:36.421611 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f4b148c9-ad63-4f22-82ac-3208e26f9edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:36.441950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b148c9-ad63-4f22-82ac-3208e26f9edf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:36.451298 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93918a34-fef8-45b1-9a0a-ac224201ed79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:36.452855 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93918a34-fef8-45b1-9a0a-ac224201ed79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:36.453374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93918a34-fef8-45b1-9a0a-ac224201ed79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:36.453896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93918a34-fef8-45b1-9a0a-ac224201ed79 admin admin] PATH_INFO: `/v3/users/91757786a7e24aa8969cbc8d7bf75073` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:36.456800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93918a34-fef8-45b1-9a0a-ac224201ed79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91757786a7e24aa8969cbc8d7bf75073)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:36.459386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93918a34-fef8-45b1-9a0a-ac224201ed79 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:36.713674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c42c5-2ec4-4f20-a904-ceea79f29f71 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:36.715712 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-190c42c5-2ec4-4f20-a904-ceea79f29f71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:36.735235 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190c42c5-2ec4-4f20-a904-ceea79f29f71 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:36.740149 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:36.740798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:36.741040 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:36.741315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068 admin admin] PATH_INFO: `/v3/projects/f9bbf8fedb084b7eb76d336b80f97aa1` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:36.742700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9bbf8fedb084b7eb76d336b80f97aa1)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:36.744695 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:36.771521 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-93918a34-fef8-45b1-9a0a-ac224201ed79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91757786a7e24aa8969cbc8d7bf75073'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:36.865746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-93918a34-fef8-45b1-9a0a-ac224201ed79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91757786a7e24aa8969cbc8d7bf75073'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:36.867542 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-93918a34-fef8-45b1-9a0a-ac224201ed79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91757786a7e24aa8969cbc8d7bf75073'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:36.868600 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-93918a34-fef8-45b1-9a0a-ac224201ed79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91757786a7e24aa8969cbc8d7bf75073'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:36.881196 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-93918a34-fef8-45b1-9a0a-ac224201ed79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91757786a7e24aa8969cbc8d7bf75073'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:36.901294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9bbf8fedb084b7eb76d336b80f97aa1', 'request_id': 'req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:36.914738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1025/2044] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:36 2022] DELETE /identity/v3/users/91757786a7e24aa8969cbc8d7bf75073 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:36.916528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bdc92-5880-4774-8323-ca498aa18e7f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:36.916828 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bdc92-5880-4774-8323-ca498aa18e7f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:36.917102 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34bdc92-5880-4774-8323-ca498aa18e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:36.918353 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9bbf8fedb084b7eb76d336b80f97aa1', 'request_id': 'req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:36.953256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e34bdc92-5880-4774-8323-ca498aa18e7f None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:36.953256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e34bdc92-5880-4774-8323-ca498aa18e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:37.034367 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1026/2045] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:37.046286 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34bdc92-5880-4774-8323-ca498aa18e7f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:37.048403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e34bdc92-5880-4774-8323-ca498aa18e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:37.110124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34bdc92-5880-4774-8323-ca498aa18e7f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:37.111030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9bbf8fedb084b7eb76d336b80f97aa1', 'request_id': 'req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:37.114739 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b3616-bf38-4c84-b7cb-5f5e08b324f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:37.115409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b3616-bf38-4c84-b7cb-5f5e08b324f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:37.115802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b3616-bf38-4c84-b7cb-5f5e08b324f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:37.116047 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b3616-bf38-4c84-b7cb-5f5e08b324f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:37.123511 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068 admin admin] The token cache is being invalidate because project f9bbf8fedb084b7eb76d336b80f97aa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:37.123832 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:37.124491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:37.125998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1020/2046] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:36 2022] DELETE /identity/v3/projects/f9bbf8fedb084b7eb76d336b80f97aa1 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:37.127115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b3616-bf38-4c84-b7cb-5f5e08b324f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:37.129633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b3616-bf38-4c84-b7cb-5f5e08b324f3 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:37.130981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:37.132690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:37.140617 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1027/2047] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:04:37.224393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6552b3f9-9ed4-4f88-a8f7-52bcd33b8068 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:37.227776 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e813bd3-6ef5-425d-8560-5768003fe370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:37.228432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e813bd3-6ef5-425d-8560-5768003fe370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:37.228713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e813bd3-6ef5-425d-8560-5768003fe370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:37.229295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e813bd3-6ef5-425d-8560-5768003fe370 admin admin] PATH_INFO: `/v3/users/ca70ddc2924f4b7ba763c54da544b11e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:37.230149 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e813bd3-6ef5-425d-8560-5768003fe370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca70ddc2924f4b7ba763c54da544b11e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:37.232575 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e813bd3-6ef5-425d-8560-5768003fe370 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:37.284782 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:37.477204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:37.700166 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4e813bd3-6ef5-425d-8560-5768003fe370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca70ddc2924f4b7ba763c54da544b11e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:37.891947 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b3616-bf38-4c84-b7cb-5f5e08b324f3 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:37.895137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7c5b3616-bf38-4c84-b7cb-5f5e08b324f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:37.911901 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b3616-bf38-4c84-b7cb-5f5e08b324f3 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:37.917744 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:37.918451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:37.918723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:37.919045 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc admin admin] PATH_INFO: `/v3/projects/3c77dd3a7d684048a3a98282d0264c09` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:37.920168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c77dd3a7d684048a3a98282d0264c09)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:37.922258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:37.930391 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4e813bd3-6ef5-425d-8560-5768003fe370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca70ddc2924f4b7ba763c54da544b11e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:37.931469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4e813bd3-6ef5-425d-8560-5768003fe370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca70ddc2924f4b7ba763c54da544b11e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:37.932573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4e813bd3-6ef5-425d-8560-5768003fe370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca70ddc2924f4b7ba763c54da544b11e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:37.951921 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4e813bd3-6ef5-425d-8560-5768003fe370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca70ddc2924f4b7ba763c54da544b11e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:37.966502 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1021/2048] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:37 2022] DELETE /identity/v3/users/ca70ddc2924f4b7ba763c54da544b11e => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:38.102504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c77dd3a7d684048a3a98282d0264c09', 'request_id': 'req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:38.108837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c77dd3a7d684048a3a98282d0264c09', 'request_id': 'req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:38.262831 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c77dd3a7d684048a3a98282d0264c09', 'request_id': 'req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:38.275375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc admin admin] The token cache is being invalidate because project 3c77dd3a7d684048a3a98282d0264c09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:38.275699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:38.276556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:38.278483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1028/2049] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:37 2022] DELETE /identity/v3/projects/3c77dd3a7d684048a3a98282d0264c09 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:38.283854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e813bd3-6ef5-425d-8560-5768003fe370 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:38.284403 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4e813bd3-6ef5-425d-8560-5768003fe370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:38.361495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e813bd3-6ef5-425d-8560-5768003fe370 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:38.430049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db95253-6f8e-4e6e-b392-ba1224d6a3eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:38.430748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db95253-6f8e-4e6e-b392-ba1224d6a3eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:38.431035 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db95253-6f8e-4e6e-b392-ba1224d6a3eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:38.431311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db95253-6f8e-4e6e-b392-ba1224d6a3eb admin admin] PATH_INFO: `/v3/users/5cabc1ac02484d87a2f2318bd6524104` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:38.432494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db95253-6f8e-4e6e-b392-ba1224d6a3eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cabc1ac02484d87a2f2318bd6524104)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:38.434483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db95253-6f8e-4e6e-b392-ba1224d6a3eb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:38.456686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:38.459407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:38.533265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fcb4e2-3c63-4a0b-8a4b-d72882a040cc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:38.537477 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c32a04-c698-493f-96f1-f33c8713d9d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:38.538152 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c32a04-c698-493f-96f1-f33c8713d9d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:38.538436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c32a04-c698-493f-96f1-f33c8713d9d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:38.538763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c32a04-c698-493f-96f1-f33c8713d9d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:38.540561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-68c32a04-c698-493f-96f1-f33c8713d9d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:38.622467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c32a04-c698-493f-96f1-f33c8713d9d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:38.623823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c32a04-c698-493f-96f1-f33c8713d9d0 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:38.636115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1029/2050] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:38 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:04:38.696982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c32a04-c698-493f-96f1-f33c8713d9d0 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:38.698501 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-68c32a04-c698-493f-96f1-f33c8713d9d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:38.713396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c32a04-c698-493f-96f1-f33c8713d9d0 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:38.718825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b612ddb-dd90-471e-9c25-462d14eed26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:38.719708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b612ddb-dd90-471e-9c25-462d14eed26b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:38.719865 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b612ddb-dd90-471e-9c25-462d14eed26b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:38.720149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b612ddb-dd90-471e-9c25-462d14eed26b admin admin] PATH_INFO: `/v3/projects/2d2c585eeef746d6b92d855a6a91be18` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:38.721561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b612ddb-dd90-471e-9c25-462d14eed26b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d2c585eeef746d6b92d855a6a91be18)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:38.724006 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b612ddb-dd90-471e-9c25-462d14eed26b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:38.734368 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:38.838083 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5db95253-6f8e-4e6e-b392-ba1224d6a3eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cabc1ac02484d87a2f2318bd6524104'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:38.924172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4b612ddb-dd90-471e-9c25-462d14eed26b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d2c585eeef746d6b92d855a6a91be18', 'request_id': 'req-4b612ddb-dd90-471e-9c25-462d14eed26b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:38.937430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4b612ddb-dd90-471e-9c25-462d14eed26b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d2c585eeef746d6b92d855a6a91be18', 'request_id': 'req-4b612ddb-dd90-471e-9c25-462d14eed26b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:38.941216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5db95253-6f8e-4e6e-b392-ba1224d6a3eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cabc1ac02484d87a2f2318bd6524104'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:38.944459 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5db95253-6f8e-4e6e-b392-ba1224d6a3eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cabc1ac02484d87a2f2318bd6524104'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:38.945320 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5db95253-6f8e-4e6e-b392-ba1224d6a3eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cabc1ac02484d87a2f2318bd6524104'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:38.963477 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5db95253-6f8e-4e6e-b392-ba1224d6a3eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cabc1ac02484d87a2f2318bd6524104'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:38.998830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1022/2051] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:38 2022] DELETE /identity/v3/users/5cabc1ac02484d87a2f2318bd6524104 => generated 0 bytes in 717 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:39.438434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4b612ddb-dd90-471e-9c25-462d14eed26b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d2c585eeef746d6b92d855a6a91be18', 'request_id': 'req-4b612ddb-dd90-471e-9c25-462d14eed26b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:39.451147 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4b612ddb-dd90-471e-9c25-462d14eed26b admin admin] The token cache is being invalidate because project 2d2c585eeef746d6b92d855a6a91be18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:39.451438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4b612ddb-dd90-471e-9c25-462d14eed26b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:39.452134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4b612ddb-dd90-471e-9c25-462d14eed26b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:39.453634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1030/2052] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:38 2022] DELETE /identity/v3/projects/2d2c585eeef746d6b92d855a6a91be18 => generated 0 bytes in 757 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:39.458768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1427c-57aa-42cc-80f2-a34f64b262a9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:39.459056 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1427c-57aa-42cc-80f2-a34f64b262a9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:39.459331 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a1427c-57aa-42cc-80f2-a34f64b262a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:39.479591 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-70a1427c-57aa-42cc-80f2-a34f64b262a9 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:39.480830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-70a1427c-57aa-42cc-80f2-a34f64b262a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:39.597447 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1023/2053] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:39.607555 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064dac84-38bd-449e-983e-a4520dd260c5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:39.608408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064dac84-38bd-449e-983e-a4520dd260c5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:39.608567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064dac84-38bd-449e-983e-a4520dd260c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:39.636785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-064dac84-38bd-449e-983e-a4520dd260c5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:39.638502 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-064dac84-38bd-449e-983e-a4520dd260c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:39.652152 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1031/2054] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:39.662580 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a1427c-57aa-42cc-80f2-a34f64b262a9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:39.673973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a1427c-57aa-42cc-80f2-a34f64b262a9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:39.680877 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc324021-35f5-4376-9f99-9504381568cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:39.681503 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc324021-35f5-4376-9f99-9504381568cd admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:39.681782 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc324021-35f5-4376-9f99-9504381568cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:39.682062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc324021-35f5-4376-9f99-9504381568cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:39.682834 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc324021-35f5-4376-9f99-9504381568cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:39.683474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc324021-35f5-4376-9f99-9504381568cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:39.685743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc324021-35f5-4376-9f99-9504381568cd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:39.698946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1024/2055] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:04:39.705722 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064dac84-38bd-449e-983e-a4520dd260c5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:39.717999 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064dac84-38bd-449e-983e-a4520dd260c5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:39.723595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f06b6-db52-42b9-a3c3-48213b2444cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:39.724451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f06b6-db52-42b9-a3c3-48213b2444cd admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:39.724906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f06b6-db52-42b9-a3c3-48213b2444cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:39.725339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761f06b6-db52-42b9-a3c3-48213b2444cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:39.726135 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f06b6-db52-42b9-a3c3-48213b2444cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:39.728383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761f06b6-db52-42b9-a3c3-48213b2444cd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:39.904268 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1032/2056] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:39 2022] POST /identity/v3/projects => generated 379 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:39.908738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc324021-35f5-4376-9f99-9504381568cd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:39.908738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bc324021-35f5-4376-9f99-9504381568cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:39.913615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f06b6-db52-42b9-a3c3-48213b2444cd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:39.935555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc324021-35f5-4376-9f99-9504381568cd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:40.030374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910f77b-e3bb-4f2b-8548-45020bc5c547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:40.032832 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910f77b-e3bb-4f2b-8548-45020bc5c547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:40.033332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910f77b-e3bb-4f2b-8548-45020bc5c547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:40.033796 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b910f77b-e3bb-4f2b-8548-45020bc5c547 admin admin] PATH_INFO: `/v3/projects/0ea87c171af840a0ace0242344273207` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:40.036774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910f77b-e3bb-4f2b-8548-45020bc5c547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ea87c171af840a0ace0242344273207)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:40.040504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b910f77b-e3bb-4f2b-8548-45020bc5c547 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:40.044332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:40.074213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761f06b6-db52-42b9-a3c3-48213b2444cd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:40.094157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177d5b6-5049-4507-92f6-20bf7896eaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:40.094157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177d5b6-5049-4507-92f6-20bf7896eaae admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:40.094157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177d5b6-5049-4507-92f6-20bf7896eaae admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:40.094157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177d5b6-5049-4507-92f6-20bf7896eaae admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:40.094579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177d5b6-5049-4507-92f6-20bf7896eaae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:40.094579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177d5b6-5049-4507-92f6-20bf7896eaae admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:40.328639 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b910f77b-e3bb-4f2b-8548-45020bc5c547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ea87c171af840a0ace0242344273207', 'request_id': 'req-b910f77b-e3bb-4f2b-8548-45020bc5c547'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:40.337426 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b910f77b-e3bb-4f2b-8548-45020bc5c547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ea87c171af840a0ace0242344273207', 'request_id': 'req-b910f77b-e3bb-4f2b-8548-45020bc5c547'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:40.340861 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1033/2057] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:39 2022] POST /identity/v3/users => generated 290 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:40.341994 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177d5b6-5049-4507-92f6-20bf7896eaae admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:40.359808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177d5b6-5049-4507-92f6-20bf7896eaae admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:40.420703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8bcd24-07d5-4c65-9f7f-abf37cf70d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:40.431626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bcd24-07d5-4c65-9f7f-abf37cf70d05 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:40.431626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bcd24-07d5-4c65-9f7f-abf37cf70d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:40.431626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8bcd24-07d5-4c65-9f7f-abf37cf70d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:40.431626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8bcd24-07d5-4c65-9f7f-abf37cf70d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:40.431626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8bcd24-07d5-4c65-9f7f-abf37cf70d05 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:40.435188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1034/2058] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:40.439204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8bcd24-07d5-4c65-9f7f-abf37cf70d05 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:40.449081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8bcd24-07d5-4c65-9f7f-abf37cf70d05 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:40.453863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27cc3d-d670-4d6c-885d-4908d085c000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:40.454496 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27cc3d-d670-4d6c-885d-4908d085c000 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:40.454778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27cc3d-d670-4d6c-885d-4908d085c000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:40.455055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27cc3d-d670-4d6c-885d-4908d085c000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:40.455740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27cc3d-d670-4d6c-885d-4908d085c000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:40.457897 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27cc3d-d670-4d6c-885d-4908d085c000 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:40.471729 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1035/2059] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:04:40.480686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27cc3d-d670-4d6c-885d-4908d085c000 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:40.491043 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27cc3d-d670-4d6c-885d-4908d085c000 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:40.495963 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414163fe-0648-48ea-858f-4c395dc9142e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:40.496682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414163fe-0648-48ea-858f-4c395dc9142e admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:40.496962 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414163fe-0648-48ea-858f-4c395dc9142e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:40.497231 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414163fe-0648-48ea-858f-4c395dc9142e admin admin] PATH_INFO: `/v3/projects/b627a92ea2fb4a5ba63763d16c7468da/users/73675882a3cf4092a390a6759f3ecad8/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:40.515126 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414163fe-0648-48ea-858f-4c395dc9142e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b627a92ea2fb4a5ba63763d16c7468da, user_id=73675882a3cf4092a390a6759f3ecad8, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:40.517817 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414163fe-0648-48ea-858f-4c395dc9142e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:40.662252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b910f77b-e3bb-4f2b-8548-45020bc5c547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ea87c171af840a0ace0242344273207', 'request_id': 'req-b910f77b-e3bb-4f2b-8548-45020bc5c547'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:40.674961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b910f77b-e3bb-4f2b-8548-45020bc5c547 admin admin] The token cache is being invalidate because project 0ea87c171af840a0ace0242344273207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:40.675405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b910f77b-e3bb-4f2b-8548-45020bc5c547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:40.676229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b910f77b-e3bb-4f2b-8548-45020bc5c547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:40.678094 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1025/2060] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:39 2022] DELETE /identity/v3/projects/0ea87c171af840a0ace0242344273207 => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:40.682359 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910f77b-e3bb-4f2b-8548-45020bc5c547 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:40.683830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b910f77b-e3bb-4f2b-8548-45020bc5c547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:40.697275 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b910f77b-e3bb-4f2b-8548-45020bc5c547 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:40.702193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4433b-9d70-4423-ace5-283526da0500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:40.702834 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4433b-9d70-4423-ace5-283526da0500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:40.703119 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4433b-9d70-4423-ace5-283526da0500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:40.703717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4433b-9d70-4423-ace5-283526da0500 admin admin] PATH_INFO: `/v3/users/616f284d65684086bcf448ef06061c66` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:40.704647 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4433b-9d70-4423-ace5-283526da0500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=616f284d65684086bcf448ef06061c66)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:40.706733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4433b-9d70-4423-ace5-283526da0500 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:40.779783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1036/2061] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:40 2022] PUT /identity/v3/projects/b627a92ea2fb4a5ba63763d16c7468da/users/73675882a3cf4092a390a6759f3ecad8/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:40.797601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a4679-dda9-491f-b707-a146dd915959 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:40.797894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a4679-dda9-491f-b707-a146dd915959 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:40.798182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32a4679-dda9-491f-b707-a146dd915959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:40.829010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b32a4679-dda9-491f-b707-a146dd915959 None None] MFA Rules not processed for user `73675882a3cf4092a390a6759f3ecad8`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:40.834092 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b32a4679-dda9-491f-b707-a146dd915959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:40.928157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1037/2062] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:40.932815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a4679-dda9-491f-b707-a146dd915959 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:40.934247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b32a4679-dda9-491f-b707-a146dd915959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:41.021999 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32a4679-dda9-491f-b707-a146dd915959 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:41.030841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337d763-a0dd-4fe9-9ee3-c14f0e348a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:41.031466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337d763-a0dd-4fe9-9ee3-c14f0e348a0f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:41.031779 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337d763-a0dd-4fe9-9ee3-c14f0e348a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:41.032054 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b337d763-a0dd-4fe9-9ee3-c14f0e348a0f admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:41.032698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337d763-a0dd-4fe9-9ee3-c14f0e348a0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:41.035899 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b337d763-a0dd-4fe9-9ee3-c14f0e348a0f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:41.045878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-55a4433b-9d70-4423-ace5-283526da0500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '616f284d65684086bcf448ef06061c66'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:41.139199 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-55a4433b-9d70-4423-ace5-283526da0500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '616f284d65684086bcf448ef06061c66'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:41.140642 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-55a4433b-9d70-4423-ace5-283526da0500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '616f284d65684086bcf448ef06061c66'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:41.148164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-55a4433b-9d70-4423-ace5-283526da0500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '616f284d65684086bcf448ef06061c66'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:41.157861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-55a4433b-9d70-4423-ace5-283526da0500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '616f284d65684086bcf448ef06061c66'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:41.174973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1026/2063] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:40 2022] DELETE /identity/v3/users/616f284d65684086bcf448ef06061c66 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:41.196235 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1038/2064] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:40 2022] POST /identity/v3/projects => generated 381 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:41.196575 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4433b-9d70-4423-ace5-283526da0500 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:41.199213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-55a4433b-9d70-4423-ace5-283526da0500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:41.304635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4433b-9d70-4423-ace5-283526da0500 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:41.377220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527de92-17ba-4da2-8132-03722d909dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:41.378419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527de92-17ba-4da2-8132-03722d909dff admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:41.378928 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527de92-17ba-4da2-8132-03722d909dff admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:41.379913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527de92-17ba-4da2-8132-03722d909dff admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:41.380738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527de92-17ba-4da2-8132-03722d909dff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:41.383217 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527de92-17ba-4da2-8132-03722d909dff admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:41.562092 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1027/2065] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:41 2022] POST /identity/v3/users => generated 297 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:41.567508 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337d763-a0dd-4fe9-9ee3-c14f0e348a0f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:41.581172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b337d763-a0dd-4fe9-9ee3-c14f0e348a0f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:41.586610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090758c-bf7a-4b67-96e5-fe9008938bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:41.587293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090758c-bf7a-4b67-96e5-fe9008938bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:41.587709 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090758c-bf7a-4b67-96e5-fe9008938bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:41.588077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8090758c-bf7a-4b67-96e5-fe9008938bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:41.588933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090758c-bf7a-4b67-96e5-fe9008938bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:41.591211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8090758c-bf7a-4b67-96e5-fe9008938bd0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:41.606505 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1039/2066] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:41.614031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527de92-17ba-4da2-8132-03722d909dff admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:41.632331 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527de92-17ba-4da2-8132-03722d909dff admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:41.639639 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa425e-7bae-4557-8b9b-bd08838fac6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:41.641670 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa425e-7bae-4557-8b9b-bd08838fac6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:41.641670 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa425e-7bae-4557-8b9b-bd08838fac6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:41.641670 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa425e-7bae-4557-8b9b-bd08838fac6d admin admin] PATH_INFO: `/v3/projects/522273ac12e143cb985e48b8896f3a08/users/74c370a37ec949c3a8c1617f593744f8/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:41.661428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa425e-7bae-4557-8b9b-bd08838fac6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=522273ac12e143cb985e48b8896f3a08, user_id=74c370a37ec949c3a8c1617f593744f8, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:41.664870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fa425e-7bae-4557-8b9b-bd08838fac6d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:41.862904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1028/2067] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:41 2022] PUT /identity/v3/projects/522273ac12e143cb985e48b8896f3a08/users/74c370a37ec949c3a8c1617f593744f8/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:41.867227 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090758c-bf7a-4b67-96e5-fe9008938bd0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:41.877428 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8090758c-bf7a-4b67-96e5-fe9008938bd0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:41.902219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:41.921847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa425e-7bae-4557-8b9b-bd08838fac6d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:41.924870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-01fa425e-7bae-4557-8b9b-bd08838fac6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:41.964045 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900b6a4-e0a3-408a-97f3-0147f15b0487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:41.964045 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900b6a4-e0a3-408a-97f3-0147f15b0487 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:41.964045 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900b6a4-e0a3-408a-97f3-0147f15b0487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:41.964045 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c900b6a4-e0a3-408a-97f3-0147f15b0487 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:41.965255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900b6a4-e0a3-408a-97f3-0147f15b0487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:41.965255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c900b6a4-e0a3-408a-97f3-0147f15b0487 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:41.974355 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1040/2068] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:41 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:41.978950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900b6a4-e0a3-408a-97f3-0147f15b0487 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:41.992327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c900b6a4-e0a3-408a-97f3-0147f15b0487 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:42.001262 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0f05d-0dd4-4034-90e3-64d083c5c6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:42.010115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0f05d-0dd4-4034-90e3-64d083c5c6ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:42.010115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0f05d-0dd4-4034-90e3-64d083c5c6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:42.010115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0f05d-0dd4-4034-90e3-64d083c5c6ef admin admin] PATH_INFO: `/v3/projects/522273ac12e143cb985e48b8896f3a08/users/74c370a37ec949c3a8c1617f593744f8/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:42.010115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0f05d-0dd4-4034-90e3-64d083c5c6ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=522273ac12e143cb985e48b8896f3a08, user_id=74c370a37ec949c3a8c1617f593744f8, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:42.012447 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0f05d-0dd4-4034-90e3-64d083c5c6ef admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:42.026585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa425e-7bae-4557-8b9b-bd08838fac6d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:42.032238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1041/2069] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:41 2022] PUT /identity/v3/projects/522273ac12e143cb985e48b8896f3a08/users/74c370a37ec949c3a8c1617f593744f8/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:42.052658 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe10348-35c4-4bb0-9f9e-e45d262d5332 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:42.055765 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe10348-35c4-4bb0-9f9e-e45d262d5332 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:42.055765 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe10348-35c4-4bb0-9f9e-e45d262d5332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:42.089197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed04246-d703-4ca7-8960-e699ddb12303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:42.089197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed04246-d703-4ca7-8960-e699ddb12303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:42.089895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed04246-d703-4ca7-8960-e699ddb12303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:42.089895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed04246-d703-4ca7-8960-e699ddb12303 admin admin] PATH_INFO: `/v3/projects/9848f41b49f141b885411aba8ccfe54e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:42.091518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed04246-d703-4ca7-8960-e699ddb12303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9848f41b49f141b885411aba8ccfe54e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:42.093837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed04246-d703-4ca7-8960-e699ddb12303 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:42.103668 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-efe10348-35c4-4bb0-9f9e-e45d262d5332 None None] MFA Rules not processed for user `74c370a37ec949c3a8c1617f593744f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:42.105809 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-efe10348-35c4-4bb0-9f9e-e45d262d5332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:42.284150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1042/2070] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:42 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:42.306161 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe10348-35c4-4bb0-9f9e-e45d262d5332 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:42.309245 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-efe10348-35c4-4bb0-9f9e-e45d262d5332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:42.337859 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6ed04246-d703-4ca7-8960-e699ddb12303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9848f41b49f141b885411aba8ccfe54e', 'request_id': 'req-6ed04246-d703-4ca7-8960-e699ddb12303'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:42.349970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6ed04246-d703-4ca7-8960-e699ddb12303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9848f41b49f141b885411aba8ccfe54e', 'request_id': 'req-6ed04246-d703-4ca7-8960-e699ddb12303'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:42.397573 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe10348-35c4-4bb0-9f9e-e45d262d5332 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:42.462759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf23e7-117e-4cd9-9765-8b91e5aa77d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:42.463376 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf23e7-117e-4cd9-9765-8b91e5aa77d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:42.463740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf23e7-117e-4cd9-9765-8b91e5aa77d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:42.464013 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf23e7-117e-4cd9-9765-8b91e5aa77d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:42.479096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6ed04246-d703-4ca7-8960-e699ddb12303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9848f41b49f141b885411aba8ccfe54e', 'request_id': 'req-6ed04246-d703-4ca7-8960-e699ddb12303'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:42.502471 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6ed04246-d703-4ca7-8960-e699ddb12303 admin admin] The token cache is being invalidate because project 9848f41b49f141b885411aba8ccfe54e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:42.503215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6ed04246-d703-4ca7-8960-e699ddb12303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:42.504880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6ed04246-d703-4ca7-8960-e699ddb12303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:42.507120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1029/2071] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:41 2022] DELETE /identity/v3/projects/9848f41b49f141b885411aba8ccfe54e => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:42.518072 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b439e-ec70-4744-8af4-c9d755754a83 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:42.518725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b439e-ec70-4744-8af4-c9d755754a83 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:42.519174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525b439e-ec70-4744-8af4-c9d755754a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:42.554988 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-525b439e-ec70-4744-8af4-c9d755754a83 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:42.555860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf23e7-117e-4cd9-9765-8b91e5aa77d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:42.557112 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-525b439e-ec70-4744-8af4-c9d755754a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:42.563705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf23e7-117e-4cd9-9765-8b91e5aa77d5 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:42.570717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1043/2072] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:42.671674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1030/2073] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:42.682613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ea2ba-f60c-4497-9991-dd1d6fa47a78 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:42.682892 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ea2ba-f60c-4497-9991-dd1d6fa47a78 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:42.683174 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5ea2ba-f60c-4497-9991-dd1d6fa47a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:42.706979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fc5ea2ba-f60c-4497-9991-dd1d6fa47a78 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:42.708485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fc5ea2ba-f60c-4497-9991-dd1d6fa47a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:42.721272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1044/2074] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:42.737790 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b439e-ec70-4744-8af4-c9d755754a83 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:42.742486 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525b439e-ec70-4744-8af4-c9d755754a83 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:42.748595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9556b92-241c-4cd2-bc0a-4361aa880134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:42.749392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9556b92-241c-4cd2-bc0a-4361aa880134 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:42.749853 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9556b92-241c-4cd2-bc0a-4361aa880134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:42.750315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9556b92-241c-4cd2-bc0a-4361aa880134 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:42.751270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9556b92-241c-4cd2-bc0a-4361aa880134 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:42.751781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9556b92-241c-4cd2-bc0a-4361aa880134 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:42.753727 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9556b92-241c-4cd2-bc0a-4361aa880134 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:42.761122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ea2ba-f60c-4497-9991-dd1d6fa47a78 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:42.762653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fc5ea2ba-f60c-4497-9991-dd1d6fa47a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:42.775185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1031/2075] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:04:42.775373 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5ea2ba-f60c-4497-9991-dd1d6fa47a78 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:42.776489 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9556b92-241c-4cd2-bc0a-4361aa880134 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:42.780242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005cdfe-4231-4a2b-8eda-15b3e237bee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:42.787336 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005cdfe-4231-4a2b-8eda-15b3e237bee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:42.787336 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005cdfe-4231-4a2b-8eda-15b3e237bee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:42.787336 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005cdfe-4231-4a2b-8eda-15b3e237bee7 admin admin] PATH_INFO: `/v3/users/ba3c05f195ee47148189a400cc1ac118` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:42.787336 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005cdfe-4231-4a2b-8eda-15b3e237bee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba3c05f195ee47148189a400cc1ac118)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:42.787336 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005cdfe-4231-4a2b-8eda-15b3e237bee7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:42.788371 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9556b92-241c-4cd2-bc0a-4361aa880134 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:42.794080 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de21d2-e76d-4291-a837-c5b8bf3a0304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:42.794766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de21d2-e76d-4291-a837-c5b8bf3a0304 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:42.795023 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de21d2-e76d-4291-a837-c5b8bf3a0304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:42.795297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de21d2-e76d-4291-a837-c5b8bf3a0304 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:42.796020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de21d2-e76d-4291-a837-c5b8bf3a0304 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:42.798030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de21d2-e76d-4291-a837-c5b8bf3a0304 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:42.948792 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1032/2076] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:42 2022] POST /identity/v3/projects => generated 385 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:42.953349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de21d2-e76d-4291-a837-c5b8bf3a0304 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:43.034137 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de21d2-e76d-4291-a837-c5b8bf3a0304 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:43.039990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6febff2-bf35-49b3-9ee1-5254743871aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:43.040647 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6febff2-bf35-49b3-9ee1-5254743871aa admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:43.041293 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6febff2-bf35-49b3-9ee1-5254743871aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:43.041575 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6febff2-bf35-49b3-9ee1-5254743871aa admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:43.042236 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6febff2-bf35-49b3-9ee1-5254743871aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:43.044386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6febff2-bf35-49b3-9ee1-5254743871aa admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:43.093699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a005cdfe-4231-4a2b-8eda-15b3e237bee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba3c05f195ee47148189a400cc1ac118'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:43.275398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a005cdfe-4231-4a2b-8eda-15b3e237bee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba3c05f195ee47148189a400cc1ac118'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:43.276023 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a005cdfe-4231-4a2b-8eda-15b3e237bee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba3c05f195ee47148189a400cc1ac118'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:43.276778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a005cdfe-4231-4a2b-8eda-15b3e237bee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba3c05f195ee47148189a400cc1ac118'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:43.276914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:43.283615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1033/2077] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:42 2022] POST /identity/v3/users => generated 293 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:43.288205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6febff2-bf35-49b3-9ee1-5254743871aa admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:43.289832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a005cdfe-4231-4a2b-8eda-15b3e237bee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba3c05f195ee47148189a400cc1ac118'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:43.290582 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f6febff2-bf35-49b3-9ee1-5254743871aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:43.305144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6febff2-bf35-49b3-9ee1-5254743871aa admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:43.311224 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ad65c-687b-48d9-99c5-ec48c24df593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:43.312095 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ad65c-687b-48d9-99c5-ec48c24df593 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:43.312378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ad65c-687b-48d9-99c5-ec48c24df593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:43.313007 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1045/2078] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:42 2022] DELETE /identity/v3/users/ba3c05f195ee47148189a400cc1ac118 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:43.313130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ad65c-687b-48d9-99c5-ec48c24df593 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:43.313467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ad65c-687b-48d9-99c5-ec48c24df593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:43.315247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ad65c-687b-48d9-99c5-ec48c24df593 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:43.319803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ee225-286d-4c2d-b83d-379c0c5d9566 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:43.320072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ee225-286d-4c2d-b83d-379c0c5d9566 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:43.320445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ee225-286d-4c2d-b83d-379c0c5d9566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:43.330640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1034/2079] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:43.335991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ad65c-687b-48d9-99c5-ec48c24df593 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:43.345557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5c1ee225-286d-4c2d-b83d-379c0c5d9566 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:43.346725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ad65c-687b-48d9-99c5-ec48c24df593 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:43.347085 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5c1ee225-286d-4c2d-b83d-379c0c5d9566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:43.368906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:43.426168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc173e8-87d1-454c-9a0c-9569d1d06316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:43.427192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc173e8-87d1-454c-9a0c-9569d1d06316 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:43.427438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc173e8-87d1-454c-9a0c-9569d1d06316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:43.427813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc173e8-87d1-454c-9a0c-9569d1d06316 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:43.428406 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc173e8-87d1-454c-9a0c-9569d1d06316 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:43.430186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc173e8-87d1-454c-9a0c-9569d1d06316 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:43.438812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1046/2080] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:43.461099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ee225-286d-4c2d-b83d-379c0c5d9566 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:43.461099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5c1ee225-286d-4c2d-b83d-379c0c5d9566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:43.461554 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1035/2081] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:43 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:43.464621 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc173e8-87d1-454c-9a0c-9569d1d06316 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:43.478085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc173e8-87d1-454c-9a0c-9569d1d06316 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:43.483430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0e966-b0a4-41b4-9fa8-33b7d8348433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:43.484433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0e966-b0a4-41b4-9fa8-33b7d8348433 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:43.484886 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0e966-b0a4-41b4-9fa8-33b7d8348433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:43.485223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e0e966-b0a4-41b4-9fa8-33b7d8348433 admin admin] PATH_INFO: `/v3/projects/a0ed0a3eace34b6ab24467e31362308f/users/32cc9d07c1e84d47b07832abd3b4a6e8/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:43.505001 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0e966-b0a4-41b4-9fa8-33b7d8348433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0ed0a3eace34b6ab24467e31362308f, user_id=32cc9d07c1e84d47b07832abd3b4a6e8, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:43.514985 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e0e966-b0a4-41b4-9fa8-33b7d8348433 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:43.536212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ee225-286d-4c2d-b83d-379c0c5d9566 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:43.539859 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f0788d-1a5f-4e48-9fb2-fe7d85e779c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:43.540460 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0788d-1a5f-4e48-9fb2-fe7d85e779c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:43.540737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0788d-1a5f-4e48-9fb2-fe7d85e779c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:43.541039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f0788d-1a5f-4e48-9fb2-fe7d85e779c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:43.550368 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0788d-1a5f-4e48-9fb2-fe7d85e779c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:43.552539 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f0788d-1a5f-4e48-9fb2-fe7d85e779c0 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:43.562363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1047/2082] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:43.617219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1036/2083] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:43 2022] PUT /identity/v3/projects/a0ed0a3eace34b6ab24467e31362308f/users/32cc9d07c1e84d47b07832abd3b4a6e8/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:43.628867 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca4753-ddbb-4434-ac2f-ab52420b52ff None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:43.629223 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca4753-ddbb-4434-ac2f-ab52420b52ff None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:43.629499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dca4753-ddbb-4434-ac2f-ab52420b52ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:43.660749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9dca4753-ddbb-4434-ac2f-ab52420b52ff None None] MFA Rules not processed for user `32cc9d07c1e84d47b07832abd3b4a6e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:43.661994 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9dca4753-ddbb-4434-ac2f-ab52420b52ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:43.733680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1048/2084] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:43.742474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0e966-b0a4-41b4-9fa8-33b7d8348433 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:43.833148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e0e966-b0a4-41b4-9fa8-33b7d8348433 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:43.848861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f643ca4-25cc-4108-8aa8-205d4e62f76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:43.849549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f643ca4-25cc-4108-8aa8-205d4e62f76f admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:43.850050 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f643ca4-25cc-4108-8aa8-205d4e62f76f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:43.850200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f643ca4-25cc-4108-8aa8-205d4e62f76f admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:43.850751 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f643ca4-25cc-4108-8aa8-205d4e62f76f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:43.852727 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f643ca4-25cc-4108-8aa8-205d4e62f76f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:44.055203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1037/2085] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:43 2022] POST /identity/v3/projects => generated 385 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:44.059744 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca4753-ddbb-4434-ac2f-ab52420b52ff None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:44.073704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dca4753-ddbb-4434-ac2f-ab52420b52ff None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:44.144559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75b337-aa8b-4d75-aa61-6f729cb5c989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:44.152635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75b337-aa8b-4d75-aa61-6f729cb5c989 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:44.152635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75b337-aa8b-4d75-aa61-6f729cb5c989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:44.152635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a75b337-aa8b-4d75-aa61-6f729cb5c989 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:44.152635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a75b337-aa8b-4d75-aa61-6f729cb5c989 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:44.152635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a75b337-aa8b-4d75-aa61-6f729cb5c989 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:44.159406 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f643ca4-25cc-4108-8aa8-205d4e62f76f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:44.161135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5f643ca4-25cc-4108-8aa8-205d4e62f76f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:44.233286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f643ca4-25cc-4108-8aa8-205d4e62f76f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:44.238844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de32326-9dda-488c-b50b-ffa6166a2eec tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c370a37ec949c3a8c1617f593744f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522273ac12e143cb985e48b8896f3a08', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:44.239734 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de32326-9dda-488c-b50b-ffa6166a2eec tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:44.240023 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de32326-9dda-488c-b50b-ffa6166a2eec tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:44.240369 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de32326-9dda-488c-b50b-ffa6166a2eec tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:44.241004 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de32326-9dda-488c-b50b-ffa6166a2eec tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:44.241311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de32326-9dda-488c-b50b-ffa6166a2eec tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:44.243006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de32326-9dda-488c-b50b-ffa6166a2eec tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:44.258417 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1038/2086] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:04:44.262728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de32326-9dda-488c-b50b-ffa6166a2eec tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:44.272853 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de32326-9dda-488c-b50b-ffa6166a2eec tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:44.277540 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37ba6d-56f6-43ab-a8c1-0ceb4ab91e8c tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c370a37ec949c3a8c1617f593744f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522273ac12e143cb985e48b8896f3a08', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:44.278122 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37ba6d-56f6-43ab-a8c1-0ceb4ab91e8c tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:44.278403 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37ba6d-56f6-43ab-a8c1-0ceb4ab91e8c tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:44.278680 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37ba6d-56f6-43ab-a8c1-0ceb4ab91e8c tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:44.279304 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37ba6d-56f6-43ab-a8c1-0ceb4ab91e8c tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:44.281217 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e37ba6d-56f6-43ab-a8c1-0ceb4ab91e8c tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:44.330683 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1049/2087] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:44 2022] POST /identity/v3/users => generated 293 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:44.332909 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75b337-aa8b-4d75-aa61-6f729cb5c989 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:44.335798 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6a75b337-aa8b-4d75-aa61-6f729cb5c989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:44.352904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a75b337-aa8b-4d75-aa61-6f729cb5c989 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:44.359301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:44.360053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:44.360205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:44.360483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406 admin admin] PATH_INFO: `/v3/projects/8e000e309ef2471096d87b38ea3c8cb6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:44.361659 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e000e309ef2471096d87b38ea3c8cb6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:44.363609 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:44.459008 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1039/2088] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:44 2022] POST /identity/v3/projects => generated 355 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:44.467681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37ba6d-56f6-43ab-a8c1-0ceb4ab91e8c tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:44.547787 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e000e309ef2471096d87b38ea3c8cb6', 'request_id': 'req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:44.562288 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e000e309ef2471096d87b38ea3c8cb6', 'request_id': 'req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:44.576899 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37ba6d-56f6-43ab-a8c1-0ceb4ab91e8c tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:44.653326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acba3c-9245-44fc-8685-fb93d9c0d782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:44.653934 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acba3c-9245-44fc-8685-fb93d9c0d782 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:44.654182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acba3c-9245-44fc-8685-fb93d9c0d782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:44.654435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15acba3c-9245-44fc-8685-fb93d9c0d782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:44.655132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acba3c-9245-44fc-8685-fb93d9c0d782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:44.656735 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15acba3c-9245-44fc-8685-fb93d9c0d782 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:44.667320 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1040/2089] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:44.668185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acba3c-9245-44fc-8685-fb93d9c0d782 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:44.703208 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e000e309ef2471096d87b38ea3c8cb6', 'request_id': 'req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:44.719290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406 admin admin] The token cache is being invalidate because project 8e000e309ef2471096d87b38ea3c8cb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:44.719698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:44.720273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:44.721794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1050/2090] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:44 2022] DELETE /identity/v3/projects/8e000e309ef2471096d87b38ea3c8cb6 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:44.722563 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:44.726065 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:44.740969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c7f9a-4033-4cc0-82c9-d5ce5b3fd406 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:44.743352 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15acba3c-9245-44fc-8685-fb93d9c0d782 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:44.747165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80093913-a075-44b1-8887-ac1d67fb8da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:44.748035 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80093913-a075-44b1-8887-ac1d67fb8da0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:44.748358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388b88d-9ba0-4152-8b33-c8fa40fd0b67 tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c370a37ec949c3a8c1617f593744f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522273ac12e143cb985e48b8896f3a08', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:44.748486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80093913-a075-44b1-8887-ac1d67fb8da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:44.748966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388b88d-9ba0-4152-8b33-c8fa40fd0b67 tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:44.749114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80093913-a075-44b1-8887-ac1d67fb8da0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:44.749409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388b88d-9ba0-4152-8b33-c8fa40fd0b67 tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:44.749526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388b88d-9ba0-4152-8b33-c8fa40fd0b67 tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:44.750051 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80093913-a075-44b1-8887-ac1d67fb8da0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:44.750174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388b88d-9ba0-4152-8b33-c8fa40fd0b67 tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:44.750449 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388b88d-9ba0-4152-8b33-c8fa40fd0b67 tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:44.752084 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388b88d-9ba0-4152-8b33-c8fa40fd0b67 tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:44.752637 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80093913-a075-44b1-8887-ac1d67fb8da0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:44.771255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1051/2091] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:44 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:44.772483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80093913-a075-44b1-8887-ac1d67fb8da0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:44.775366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-80093913-a075-44b1-8887-ac1d67fb8da0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:44.779720 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1041/2092] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:04:44.780961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388b88d-9ba0-4152-8b33-c8fa40fd0b67 tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:44.796722 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80093913-a075-44b1-8887-ac1d67fb8da0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:44.797970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388b88d-9ba0-4152-8b33-c8fa40fd0b67 tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:44.804171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96a0b8-9088-4fb3-912c-206fdf034379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:44.805147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80016786-58e9-45ac-a42b-1e43fa76aca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:44.805334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96a0b8-9088-4fb3-912c-206fdf034379 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:44.805334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96a0b8-9088-4fb3-912c-206fdf034379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:44.805486 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80016786-58e9-45ac-a42b-1e43fa76aca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:44.805877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96a0b8-9088-4fb3-912c-206fdf034379 admin admin] PATH_INFO: `/v3/users/eb7dcd4c8d4d486bb28ce51a816ffe21` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:44.806038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80016786-58e9-45ac-a42b-1e43fa76aca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:44.806141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80016786-58e9-45ac-a42b-1e43fa76aca3 admin admin] PATH_INFO: `/v3/projects/4ac92811ebcb4b68a3d889e7aad244f9/users/bffac5553f734e82918d9e8f9087ea7b/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:44.807188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e96a0b8-9088-4fb3-912c-206fdf034379 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb7dcd4c8d4d486bb28ce51a816ffe21)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:44.809545 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e96a0b8-9088-4fb3-912c-206fdf034379 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:44.825410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80016786-58e9-45ac-a42b-1e43fa76aca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ac92811ebcb4b68a3d889e7aad244f9, user_id=bffac5553f734e82918d9e8f9087ea7b, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:44.828499 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80016786-58e9-45ac-a42b-1e43fa76aca3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:44.946237 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1042/2093] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:44 2022] PUT /identity/v3/projects/4ac92811ebcb4b68a3d889e7aad244f9/users/bffac5553f734e82918d9e8f9087ea7b/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:44.947465 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80016786-58e9-45ac-a42b-1e43fa76aca3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:44.949858 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-80016786-58e9-45ac-a42b-1e43fa76aca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:45.025117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80016786-58e9-45ac-a42b-1e43fa76aca3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:45.030051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74c370a37ec949c3a8c1617f593744f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '522273ac12e143cb985e48b8896f3a08', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:45.030535 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:45.030818 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:45.031192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] PATH_INFO: `/v3/projects/5346ebcbeac940d4b0cb56f9c14b7443` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:45.032403 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5346ebcbeac940d4b0cb56f9c14b7443)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:45.039980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:45.111905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e96a0b8-9088-4fb3-912c-206fdf034379 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb7dcd4c8d4d486bb28ce51a816ffe21'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:45.149361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5346ebcbeac940d4b0cb56f9c14b7443', 'request_id': 'req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:45.156314 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5346ebcbeac940d4b0cb56f9c14b7443', 'request_id': 'req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:45.210950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e96a0b8-9088-4fb3-912c-206fdf034379 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb7dcd4c8d4d486bb28ce51a816ffe21'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:45.212200 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e96a0b8-9088-4fb3-912c-206fdf034379 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb7dcd4c8d4d486bb28ce51a816ffe21'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:45.212874 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e96a0b8-9088-4fb3-912c-206fdf034379 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb7dcd4c8d4d486bb28ce51a816ffe21'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:45.223358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1e96a0b8-9088-4fb3-912c-206fdf034379 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb7dcd4c8d4d486bb28ce51a816ffe21'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:45.247700 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1052/2094] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:44 2022] DELETE /identity/v3/users/eb7dcd4c8d4d486bb28ce51a816ffe21 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:45.249482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b99b2-6b20-4ba2-9306-4e283953298a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:45.249827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b99b2-6b20-4ba2-9306-4e283953298a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:45.257829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723b99b2-6b20-4ba2-9306-4e283953298a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:45.276901 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5346ebcbeac940d4b0cb56f9c14b7443', 'request_id': 'req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:45.297710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-723b99b2-6b20-4ba2-9306-4e283953298a None None] MFA Rules not processed for user `bffac5553f734e82918d9e8f9087ea7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:45.298824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-723b99b2-6b20-4ba2-9306-4e283953298a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:45.308637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] The token cache is being invalidate because project 5346ebcbeac940d4b0cb56f9c14b7443 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:45.309216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:45.310438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:45.313638 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1043/2095] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:44 2022] DELETE /identity/v3/projects/5346ebcbeac940d4b0cb56f9c14b7443 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:45.407307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1053/2096] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:45.412182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:45.413745 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:45.496899 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193f3cc-96f7-49dc-828e-55c5c2d3fabc tempest-VolumeQuotaClassesTest-1108310080 tempest-VolumeQuotaClassesTest-1108310080-project-admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:45.576998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc56204-72d1-49ee-8b36-c5050f7431d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:45.578155 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc56204-72d1-49ee-8b36-c5050f7431d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:45.579057 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc56204-72d1-49ee-8b36-c5050f7431d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:45.579560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc56204-72d1-49ee-8b36-c5050f7431d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:45.580662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc56204-72d1-49ee-8b36-c5050f7431d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:45.585121 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc56204-72d1-49ee-8b36-c5050f7431d7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:45.795303 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1044/2097] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:45 2022] POST /identity/v3/projects => generated 383 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:45.797193 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b99b2-6b20-4ba2-9306-4e283953298a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:45.814583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723b99b2-6b20-4ba2-9306-4e283953298a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:45.898454 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0ddfe-d796-4587-a5d0-ce6687c4a343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:45.899210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0ddfe-d796-4587-a5d0-ce6687c4a343 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:45.899672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0ddfe-d796-4587-a5d0-ce6687c4a343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:45.899948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d0ddfe-d796-4587-a5d0-ce6687c4a343 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:45.900771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0ddfe-d796-4587-a5d0-ce6687c4a343 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:45.903227 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d0ddfe-d796-4587-a5d0-ce6687c4a343 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:46.080112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:46.089323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc56204-72d1-49ee-8b36-c5050f7431d7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:46.089507 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1054/2098] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:45 2022] POST /identity/v3/users => generated 298 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:46.089596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0cc56204-72d1-49ee-8b36-c5050f7431d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:46.097967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0ddfe-d796-4587-a5d0-ce6687c4a343 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:46.102052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d0ddfe-d796-4587-a5d0-ce6687c4a343 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:46.107424 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ac54c-80d9-4f25-aa59-b2b4770432d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:46.107764 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc56204-72d1-49ee-8b36-c5050f7431d7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:46.108094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ac54c-80d9-4f25-aa59-b2b4770432d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:46.108488 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ac54c-80d9-4f25-aa59-b2b4770432d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:46.108692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ac54c-80d9-4f25-aa59-b2b4770432d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:46.109583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ac54c-80d9-4f25-aa59-b2b4770432d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:46.111841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ac54c-80d9-4f25-aa59-b2b4770432d1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:46.115601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc20f56-4d85-4f35-8b84-6487605de458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:46.116306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc20f56-4d85-4f35-8b84-6487605de458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:46.116660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc20f56-4d85-4f35-8b84-6487605de458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:46.116896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc20f56-4d85-4f35-8b84-6487605de458 admin admin] PATH_INFO: `/v3/projects/26bdd9d6e2e74909ae1030a4e845ac53` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:46.118159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc20f56-4d85-4f35-8b84-6487605de458 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26bdd9d6e2e74909ae1030a4e845ac53)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:46.120429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc20f56-4d85-4f35-8b84-6487605de458 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:46.123925 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1055/2099] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:46.128230 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ac54c-80d9-4f25-aa59-b2b4770432d1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:46.139435 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ac54c-80d9-4f25-aa59-b2b4770432d1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:46.144991 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8feb52-f7fe-48b7-b966-b31a60e2f262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:46.145668 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8feb52-f7fe-48b7-b966-b31a60e2f262 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:46.149094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8feb52-f7fe-48b7-b966-b31a60e2f262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:46.149094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8feb52-f7fe-48b7-b966-b31a60e2f262 admin admin] PATH_INFO: `/v3/projects/e676907015fa483695dbd661846ad887/users/54c4bde5b73e448fab87831d727d4f0c/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:46.162189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8feb52-f7fe-48b7-b966-b31a60e2f262 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e676907015fa483695dbd661846ad887, user_id=54c4bde5b73e448fab87831d727d4f0c, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:46.165087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8feb52-f7fe-48b7-b966-b31a60e2f262 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:46.441946 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1056/2100] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:46 2022] PUT /identity/v3/projects/e676907015fa483695dbd661846ad887/users/54c4bde5b73e448fab87831d727d4f0c/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:46.451199 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8feb52-f7fe-48b7-b966-b31a60e2f262 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:46.451674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2cc20f56-4d85-4f35-8b84-6487605de458 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26bdd9d6e2e74909ae1030a4e845ac53', 'request_id': 'req-2cc20f56-4d85-4f35-8b84-6487605de458'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:46.463459 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2cc20f56-4d85-4f35-8b84-6487605de458 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26bdd9d6e2e74909ae1030a4e845ac53', 'request_id': 'req-2cc20f56-4d85-4f35-8b84-6487605de458'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:46.529639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8feb52-f7fe-48b7-b966-b31a60e2f262 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:46.594172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04053958-20da-462e-b4b8-e82f3cf290f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:46.594730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04053958-20da-462e-b4b8-e82f3cf290f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:46.595030 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04053958-20da-462e-b4b8-e82f3cf290f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:46.595357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04053958-20da-462e-b4b8-e82f3cf290f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:46.596038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04053958-20da-462e-b4b8-e82f3cf290f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:46.597660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04053958-20da-462e-b4b8-e82f3cf290f4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:46.603560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2cc20f56-4d85-4f35-8b84-6487605de458 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26bdd9d6e2e74909ae1030a4e845ac53', 'request_id': 'req-2cc20f56-4d85-4f35-8b84-6487605de458'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:46.608514 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1057/2101] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:46 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:04:46.609504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04053958-20da-462e-b4b8-e82f3cf290f4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:46.611186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-04053958-20da-462e-b4b8-e82f3cf290f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:46.617193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2cc20f56-4d85-4f35-8b84-6487605de458 admin admin] The token cache is being invalidate because project 26bdd9d6e2e74909ae1030a4e845ac53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:46.617489 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2cc20f56-4d85-4f35-8b84-6487605de458 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:46.618167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2cc20f56-4d85-4f35-8b84-6487605de458 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:46.619948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1045/2102] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:46 2022] DELETE /identity/v3/projects/26bdd9d6e2e74909ae1030a4e845ac53 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:46.626983 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc20f56-4d85-4f35-8b84-6487605de458 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:46.627129 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04053958-20da-462e-b4b8-e82f3cf290f4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:46.627230 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2cc20f56-4d85-4f35-8b84-6487605de458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:46.635867 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd37ca31-f1d7-4e12-ab39-c3419480fd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:46.636526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37ca31-f1d7-4e12-ab39-c3419480fd2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:46.636803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37ca31-f1d7-4e12-ab39-c3419480fd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:46.637086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd37ca31-f1d7-4e12-ab39-c3419480fd2f admin admin] PATH_INFO: `/v3/users/73675882a3cf4092a390a6759f3ecad8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:46.638245 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37ca31-f1d7-4e12-ab39-c3419480fd2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=73675882a3cf4092a390a6759f3ecad8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:46.640256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd37ca31-f1d7-4e12-ab39-c3419480fd2f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:46.642943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc20f56-4d85-4f35-8b84-6487605de458 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:46.659229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f36010-d33d-41b3-879f-0e69d73ca84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:46.660005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f36010-d33d-41b3-879f-0e69d73ca84a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:46.660410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f36010-d33d-41b3-879f-0e69d73ca84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:46.660768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f36010-d33d-41b3-879f-0e69d73ca84a admin admin] PATH_INFO: `/v3/projects/e676907015fa483695dbd661846ad887/users/54c4bde5b73e448fab87831d727d4f0c/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:46.663254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f36010-d33d-41b3-879f-0e69d73ca84a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e676907015fa483695dbd661846ad887, user_id=54c4bde5b73e448fab87831d727d4f0c, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:46.666587 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f36010-d33d-41b3-879f-0e69d73ca84a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:46.684430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1046/2103] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:46 2022] PUT /identity/v3/projects/e676907015fa483695dbd661846ad887/users/54c4bde5b73e448fab87831d727d4f0c/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:46.686450 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac63ac-0a2e-47e5-bbd9-71d63d439c49 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:46.686865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac63ac-0a2e-47e5-bbd9-71d63d439c49 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:46.687529 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ac63ac-0a2e-47e5-bbd9-71d63d439c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:46.717798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e7ac63ac-0a2e-47e5-bbd9-71d63d439c49 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:46.719000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e7ac63ac-0a2e-47e5-bbd9-71d63d439c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:46.805617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1047/2104] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:46.807256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95806dd-8838-42e1-92f4-a4c9fafce0a4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:46.807543 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95806dd-8838-42e1-92f4-a4c9fafce0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:46.807841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95806dd-8838-42e1-92f4-a4c9fafce0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:46.837517 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b95806dd-8838-42e1-92f4-a4c9fafce0a4 None None] MFA Rules not processed for user `54c4bde5b73e448fab87831d727d4f0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:46.838765 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b95806dd-8838-42e1-92f4-a4c9fafce0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:46.925034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1048/2105] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:46 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:46.926567 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe69c0-b901-4ced-8aa4-976ee4e91dfb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:46.927380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe69c0-b901-4ced-8aa4-976ee4e91dfb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:46.927673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe69c0-b901-4ced-8aa4-976ee4e91dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:46.952640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b5fe69c0-b901-4ced-8aa4-976ee4e91dfb None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:46.957199 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b5fe69c0-b901-4ced-8aa4-976ee4e91dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:46.963876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1049/2106] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:46.965428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe69c0-b901-4ced-8aa4-976ee4e91dfb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:46.966752 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b5fe69c0-b901-4ced-8aa4-976ee4e91dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:47.017906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bd37ca31-f1d7-4e12-ab39-c3419480fd2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '73675882a3cf4092a390a6759f3ecad8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:47.046527 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe69c0-b901-4ced-8aa4-976ee4e91dfb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:47.051161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c14d3-ba25-4e2b-987a-68f4abd0c545 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:47.051837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c14d3-ba25-4e2b-987a-68f4abd0c545 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:47.052118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c14d3-ba25-4e2b-987a-68f4abd0c545 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:47.052390 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c14d3-ba25-4e2b-987a-68f4abd0c545 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:47.054262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9c2c14d3-ba25-4e2b-987a-68f4abd0c545 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:47.107292 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bd37ca31-f1d7-4e12-ab39-c3419480fd2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '73675882a3cf4092a390a6759f3ecad8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:47.108475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bd37ca31-f1d7-4e12-ab39-c3419480fd2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '73675882a3cf4092a390a6759f3ecad8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:47.109466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bd37ca31-f1d7-4e12-ab39-c3419480fd2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '73675882a3cf4092a390a6759f3ecad8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:47.119152 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bd37ca31-f1d7-4e12-ab39-c3419480fd2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '73675882a3cf4092a390a6759f3ecad8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:47.124234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c14d3-ba25-4e2b-987a-68f4abd0c545 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:47.125966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c14d3-ba25-4e2b-987a-68f4abd0c545 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:47.136543 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1050/2107] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:47.137394 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c14d3-ba25-4e2b-987a-68f4abd0c545 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:47.140358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1058/2108] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:46 2022] DELETE /identity/v3/users/73675882a3cf4092a390a6759f3ecad8 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:47.145098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfbfd8-36d3-45d9-a807-4a2b6c9f3bec None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:47.145383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfbfd8-36d3-45d9-a807-4a2b6c9f3bec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:47.145866 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdfbfd8-36d3-45d9-a807-4a2b6c9f3bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:47.149304 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c14d3-ba25-4e2b-987a-68f4abd0c545 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:47.167226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7cdfbfd8-36d3-45d9-a807-4a2b6c9f3bec None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:47.168501 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7cdfbfd8-36d3-45d9-a807-4a2b6c9f3bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:47.238483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2e2c1-99b9-4b2c-aa1b-236cf05fbcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:47.239916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2e2c1-99b9-4b2c-aa1b-236cf05fbcbd admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:47.240413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2e2c1-99b9-4b2c-aa1b-236cf05fbcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:47.240967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd2e2c1-99b9-4b2c-aa1b-236cf05fbcbd admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:47.242073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2e2c1-99b9-4b2c-aa1b-236cf05fbcbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:47.242613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2e2c1-99b9-4b2c-aa1b-236cf05fbcbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:47.245531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd2e2c1-99b9-4b2c-aa1b-236cf05fbcbd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:47.251388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1059/2109] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:47.258243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfbfd8-36d3-45d9-a807-4a2b6c9f3bec None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:47.260364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7cdfbfd8-36d3-45d9-a807-4a2b6c9f3bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:47.267902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1051/2110] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:04:47.273228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2e2c1-99b9-4b2c-aa1b-236cf05fbcbd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:47.275886 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7cd2e2c1-99b9-4b2c-aa1b-236cf05fbcbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:47.295960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd2e2c1-99b9-4b2c-aa1b-236cf05fbcbd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:47.302506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42a6d9-2148-483d-9142-c3ab426d48c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:47.303294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42a6d9-2148-483d-9142-c3ab426d48c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:47.303679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42a6d9-2148-483d-9142-c3ab426d48c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:47.304112 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c42a6d9-2148-483d-9142-c3ab426d48c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:47.305593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42a6d9-2148-483d-9142-c3ab426d48c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:47.308042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c42a6d9-2148-483d-9142-c3ab426d48c9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:47.332857 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdfbfd8-36d3-45d9-a807-4a2b6c9f3bec None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:47.337518 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f43633-eb31-4908-ab86-64c30ca1bb6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:47.338195 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f43633-eb31-4908-ab86-64c30ca1bb6d service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:47.338480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f43633-eb31-4908-ab86-64c30ca1bb6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:47.338763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f43633-eb31-4908-ab86-64c30ca1bb6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:47.351127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f43633-eb31-4908-ab86-64c30ca1bb6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:47.354315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f43633-eb31-4908-ab86-64c30ca1bb6d service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:47.366391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1060/2111] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:04:47.481914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1052/2112] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:47 2022] POST /identity/v3/projects => generated 365 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:47.484180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f43633-eb31-4908-ab86-64c30ca1bb6d service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:47.496299 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:47.498252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f43633-eb31-4908-ab86-64c30ca1bb6d service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:47.599429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3661ba-db8a-4e30-91b9-dd6da2985eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:47.600178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3661ba-db8a-4e30-91b9-dd6da2985eba admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:47.600476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3661ba-db8a-4e30-91b9-dd6da2985eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:47.601125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3661ba-db8a-4e30-91b9-dd6da2985eba admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:47.601884 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3661ba-db8a-4e30-91b9-dd6da2985eba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:47.604335 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3661ba-db8a-4e30-91b9-dd6da2985eba admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:47.823315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1061/2113] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:47 2022] POST /identity/v3/users => generated 283 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:47.825562 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42a6d9-2148-483d-9142-c3ab426d48c9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:47.842416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c42a6d9-2148-483d-9142-c3ab426d48c9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:47.849052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271d064-a740-4808-b6bd-599ff87a6f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:47.849990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271d064-a740-4808-b6bd-599ff87a6f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:47.850536 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271d064-a740-4808-b6bd-599ff87a6f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:47.850974 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4271d064-a740-4808-b6bd-599ff87a6f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:47.852201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271d064-a740-4808-b6bd-599ff87a6f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:47.855124 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4271d064-a740-4808-b6bd-599ff87a6f67 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:47.869642 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1053/2114] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:04:47.875475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3661ba-db8a-4e30-91b9-dd6da2985eba admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:47.897108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3661ba-db8a-4e30-91b9-dd6da2985eba admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:47.906593 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb3b67-4a50-4d75-bdf0-4d2548aa752d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:47.907217 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb3b67-4a50-4d75-bdf0-4d2548aa752d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:47.907687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb3b67-4a50-4d75-bdf0-4d2548aa752d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:47.907964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb3b67-4a50-4d75-bdf0-4d2548aa752d admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:47.908744 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb3b67-4a50-4d75-bdf0-4d2548aa752d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:47.910811 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb3b67-4a50-4d75-bdf0-4d2548aa752d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:47.922482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1062/2115] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:47.928697 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271d064-a740-4808-b6bd-599ff87a6f67 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:47.939789 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4271d064-a740-4808-b6bd-599ff87a6f67 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:47.945594 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1832c6-0cc2-4dd5-b2e5-6024f8dfe033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:47.946763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1832c6-0cc2-4dd5-b2e5-6024f8dfe033 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:47.947444 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1832c6-0cc2-4dd5-b2e5-6024f8dfe033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:47.947581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1832c6-0cc2-4dd5-b2e5-6024f8dfe033 admin admin] PATH_INFO: `/v3/projects/4f285741e21a4c5a904058e941059e1f/users/198fac9935ba44cfaaa7a17835f5aae3/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:47.963861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1832c6-0cc2-4dd5-b2e5-6024f8dfe033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f285741e21a4c5a904058e941059e1f, user_id=198fac9935ba44cfaaa7a17835f5aae3, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:47.966541 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1832c6-0cc2-4dd5-b2e5-6024f8dfe033 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:48.022048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb3b67-4a50-4d75-bdf0-4d2548aa752d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:48.024259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bcdb3b67-4a50-4d75-bdf0-4d2548aa752d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:48.039411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb3b67-4a50-4d75-bdf0-4d2548aa752d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:48.044668 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:48.045313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:48.045598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:48.045872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670 admin admin] PATH_INFO: `/v3/projects/b627a92ea2fb4a5ba63763d16c7468da` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:48.047029 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b627a92ea2fb4a5ba63763d16c7468da)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:48.049043 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:48.147641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1054/2116] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:47 2022] PUT /identity/v3/projects/4f285741e21a4c5a904058e941059e1f/users/198fac9935ba44cfaaa7a17835f5aae3/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:48.163607 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66e829-8cf5-41b3-976c-b8150a349c49 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:48.163920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66e829-8cf5-41b3-976c-b8150a349c49 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:48.165024 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66e829-8cf5-41b3-976c-b8150a349c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:48.199420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b627a92ea2fb4a5ba63763d16c7468da', 'request_id': 'req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:48.200324 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2f66e829-8cf5-41b3-976c-b8150a349c49 None None] MFA Rules not processed for user `198fac9935ba44cfaaa7a17835f5aae3`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:48.202405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2f66e829-8cf5-41b3-976c-b8150a349c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:48.208753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b627a92ea2fb4a5ba63763d16c7468da', 'request_id': 'req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:48.307841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1055/2117] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:48.311403 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66e829-8cf5-41b3-976c-b8150a349c49 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:48.352481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b627a92ea2fb4a5ba63763d16c7468da', 'request_id': 'req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:48.368647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670 admin admin] The token cache is being invalidate because project b627a92ea2fb4a5ba63763d16c7468da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:48.369094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:48.370406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:48.372465 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1063/2118] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:48 2022] DELETE /identity/v3/projects/b627a92ea2fb4a5ba63763d16c7468da => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:48.377747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:48.379849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:48.384055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66e829-8cf5-41b3-976c-b8150a349c49 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:48.392217 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d5bd3-d0eb-440f-8684-7f21f09fd0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:48.392217 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d5bd3-d0eb-440f-8684-7f21f09fd0cb admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:48.392217 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d5bd3-d0eb-440f-8684-7f21f09fd0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:48.392217 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567d5bd3-d0eb-440f-8684-7f21f09fd0cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:48.393001 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d5bd3-d0eb-440f-8684-7f21f09fd0cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:48.394965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567d5bd3-d0eb-440f-8684-7f21f09fd0cb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:48.419231 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e5c3a9-3933-487b-b0c4-3ee53fc3b670 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:48.429203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79decf6-96a7-4859-874f-9e3b2e04c563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:48.429203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79decf6-96a7-4859-874f-9e3b2e04c563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:48.429203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79decf6-96a7-4859-874f-9e3b2e04c563 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:48.429203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79decf6-96a7-4859-874f-9e3b2e04c563 admin admin] PATH_INFO: `/v3/users/74c370a37ec949c3a8c1617f593744f8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:48.430415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79decf6-96a7-4859-874f-9e3b2e04c563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74c370a37ec949c3a8c1617f593744f8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:48.433525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79decf6-96a7-4859-874f-9e3b2e04c563 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:48.642291 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1056/2119] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:48 2022] POST /identity/v3/projects => generated 363 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:48.643626 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d5bd3-d0eb-440f-8684-7f21f09fd0cb admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:48.645720 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-567d5bd3-d0eb-440f-8684-7f21f09fd0cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:48.746206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567d5bd3-d0eb-440f-8684-7f21f09fd0cb admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:48.757871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd38f72-d000-49f0-bf31-e5c3a6b69906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:48.759006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd38f72-d000-49f0-bf31-e5c3a6b69906 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:48.759441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd38f72-d000-49f0-bf31-e5c3a6b69906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:48.760349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd38f72-d000-49f0-bf31-e5c3a6b69906 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:48.761349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd38f72-d000-49f0-bf31-e5c3a6b69906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:48.765060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd38f72-d000-49f0-bf31-e5c3a6b69906 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:48.834045 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f79decf6-96a7-4859-874f-9e3b2e04c563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74c370a37ec949c3a8c1617f593744f8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:48.882069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1057/2120] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:48 2022] POST /identity/v3/users => generated 288 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:48.886743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd38f72-d000-49f0-bf31-e5c3a6b69906 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:48.902261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd38f72-d000-49f0-bf31-e5c3a6b69906 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:48.908281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a819cc-ad5d-49ce-a27c-90b21871a6f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:48.909341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a819cc-ad5d-49ce-a27c-90b21871a6f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:48.909341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a819cc-ad5d-49ce-a27c-90b21871a6f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:48.910163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a819cc-ad5d-49ce-a27c-90b21871a6f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:48.910731 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a819cc-ad5d-49ce-a27c-90b21871a6f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:48.912373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a819cc-ad5d-49ce-a27c-90b21871a6f7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:48.930389 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1058/2121] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:48.933628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a819cc-ad5d-49ce-a27c-90b21871a6f7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:48.946588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a819cc-ad5d-49ce-a27c-90b21871a6f7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:48.952145 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da29b4-ab98-439b-a5bd-f0ee5c2a73ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:48.953089 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da29b4-ab98-439b-a5bd-f0ee5c2a73ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:48.953427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da29b4-ab98-439b-a5bd-f0ee5c2a73ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:48.953796 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da29b4-ab98-439b-a5bd-f0ee5c2a73ce admin admin] PATH_INFO: `/v3/projects/b987abb21feb4024888c0353450fb058/users/b06aba66339d47b1be1876a222e9cf2d/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:48.953968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f79decf6-96a7-4859-874f-9e3b2e04c563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74c370a37ec949c3a8c1617f593744f8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:48.955190 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f79decf6-96a7-4859-874f-9e3b2e04c563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74c370a37ec949c3a8c1617f593744f8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:48.956484 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f79decf6-96a7-4859-874f-9e3b2e04c563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74c370a37ec949c3a8c1617f593744f8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:48.966677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f79decf6-96a7-4859-874f-9e3b2e04c563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74c370a37ec949c3a8c1617f593744f8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:48.975045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da29b4-ab98-439b-a5bd-f0ee5c2a73ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=b987abb21feb4024888c0353450fb058, user_id=b06aba66339d47b1be1876a222e9cf2d, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:48.977882 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da29b4-ab98-439b-a5bd-f0ee5c2a73ce admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:48.990574 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1064/2122] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:48 2022] DELETE /identity/v3/users/74c370a37ec949c3a8c1617f593744f8 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:49.126082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1059/2123] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:48 2022] PUT /identity/v3/projects/b987abb21feb4024888c0353450fb058/users/b06aba66339d47b1be1876a222e9cf2d/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:49.131782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79decf6-96a7-4859-874f-9e3b2e04c563 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:49.133247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f79decf6-96a7-4859-874f-9e3b2e04c563 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:49.224570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79decf6-96a7-4859-874f-9e3b2e04c563 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:49.319796 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a28c33f-a239-4d8a-bc43-f395eca0bed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:49.320966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28c33f-a239-4d8a-bc43-f395eca0bed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:49.321354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28c33f-a239-4d8a-bc43-f395eca0bed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:49.321848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a28c33f-a239-4d8a-bc43-f395eca0bed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:49.323089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28c33f-a239-4d8a-bc43-f395eca0bed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:49.325869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a28c33f-a239-4d8a-bc43-f395eca0bed1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:49.352000 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1065/2124] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:49 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:49.354224 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da29b4-ab98-439b-a5bd-f0ee5c2a73ce admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:49.372016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da29b4-ab98-439b-a5bd-f0ee5c2a73ce admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:49.383886 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b0ab0-96d4-4790-bf8b-82c283ffbe02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:49.384575 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b0ab0-96d4-4790-bf8b-82c283ffbe02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:49.385329 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b0ab0-96d4-4790-bf8b-82c283ffbe02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:49.385774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583b0ab0-96d4-4790-bf8b-82c283ffbe02 admin admin] PATH_INFO: `/v3/projects/b987abb21feb4024888c0353450fb058/users/b06aba66339d47b1be1876a222e9cf2d/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:49.388283 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b0ab0-96d4-4790-bf8b-82c283ffbe02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b987abb21feb4024888c0353450fb058, user_id=b06aba66339d47b1be1876a222e9cf2d, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:49.391418 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583b0ab0-96d4-4790-bf8b-82c283ffbe02 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:49.408610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1060/2125] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:49 2022] PUT /identity/v3/projects/b987abb21feb4024888c0353450fb058/users/b06aba66339d47b1be1876a222e9cf2d/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:49.425303 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1b319-8866-4534-8be5-b597c30001e3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:49.425598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1b319-8866-4534-8be5-b597c30001e3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:49.425881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a1b319-8866-4534-8be5-b597c30001e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:49.462077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b5a1b319-8866-4534-8be5-b597c30001e3 None None] MFA Rules not processed for user `b06aba66339d47b1be1876a222e9cf2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:49.463396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b5a1b319-8866-4534-8be5-b597c30001e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:49.558533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1066/2126] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:49 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:49.571963 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b0ab0-96d4-4790-bf8b-82c283ffbe02 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:49.577441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-583b0ab0-96d4-4790-bf8b-82c283ffbe02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:49.639423 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1b319-8866-4534-8be5-b597c30001e3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:49.641705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b5a1b319-8866-4534-8be5-b597c30001e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:49.697026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583b0ab0-96d4-4790-bf8b-82c283ffbe02 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:49.703082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208ee30-2df5-49af-8dc0-13632d4fc719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:49.704126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208ee30-2df5-49af-8dc0-13632d4fc719 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:49.704882 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208ee30-2df5-49af-8dc0-13632d4fc719 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:49.705011 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208ee30-2df5-49af-8dc0-13632d4fc719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:49.726438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a1b319-8866-4534-8be5-b597c30001e3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:49.727439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208ee30-2df5-49af-8dc0-13632d4fc719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:49.728436 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208ee30-2df5-49af-8dc0-13632d4fc719 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:49.736853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093d1e23-de88-4270-803f-c5d0bcb8ae4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:49.737629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d1e23-de88-4270-803f-c5d0bcb8ae4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:49.738082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d1e23-de88-4270-803f-c5d0bcb8ae4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:49.738356 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093d1e23-de88-4270-803f-c5d0bcb8ae4f admin admin] PATH_INFO: `/v3/projects/522273ac12e143cb985e48b8896f3a08` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:49.739540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d1e23-de88-4270-803f-c5d0bcb8ae4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=522273ac12e143cb985e48b8896f3a08)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:49.741983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093d1e23-de88-4270-803f-c5d0bcb8ae4f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:49.749543 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1061/2127] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:04:50.236785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-093d1e23-de88-4270-803f-c5d0bcb8ae4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '522273ac12e143cb985e48b8896f3a08', 'request_id': 'req-093d1e23-de88-4270-803f-c5d0bcb8ae4f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:50.236785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-093d1e23-de88-4270-803f-c5d0bcb8ae4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '522273ac12e143cb985e48b8896f3a08', 'request_id': 'req-093d1e23-de88-4270-803f-c5d0bcb8ae4f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:50.399873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:50.465910 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-093d1e23-de88-4270-803f-c5d0bcb8ae4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '522273ac12e143cb985e48b8896f3a08', 'request_id': 'req-093d1e23-de88-4270-803f-c5d0bcb8ae4f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:50.486369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-093d1e23-de88-4270-803f-c5d0bcb8ae4f admin admin] The token cache is being invalidate because project 522273ac12e143cb985e48b8896f3a08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:50.486369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-093d1e23-de88-4270-803f-c5d0bcb8ae4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:50.486369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-093d1e23-de88-4270-803f-c5d0bcb8ae4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:50.489207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1067/2128] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:49 2022] DELETE /identity/v3/projects/522273ac12e143cb985e48b8896f3a08 => generated 0 bytes in 850 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:50.495205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7380d92-be59-4e68-b72b-88adac49ae4e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:50.495943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7380d92-be59-4e68-b72b-88adac49ae4e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:50.496471 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7380d92-be59-4e68-b72b-88adac49ae4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:50.529800 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c7380d92-be59-4e68-b72b-88adac49ae4e None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:50.530321 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c7380d92-be59-4e68-b72b-88adac49ae4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:50.599067 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1062/2129] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:50.606926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd17447-14a1-4c63-91d6-89770ea2f4f7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:50.607210 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd17447-14a1-4c63-91d6-89770ea2f4f7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:50.607483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd17447-14a1-4c63-91d6-89770ea2f4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:50.627681 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4fd17447-14a1-4c63-91d6-89770ea2f4f7 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:50.629443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4fd17447-14a1-4c63-91d6-89770ea2f4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:50.639970 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1068/2130] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:50.645028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7380d92-be59-4e68-b72b-88adac49ae4e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:50.655256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7380d92-be59-4e68-b72b-88adac49ae4e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:50.660666 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6d5de-592b-401a-bfb1-6060520c875f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:50.661279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6d5de-592b-401a-bfb1-6060520c875f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:50.661557 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6d5de-592b-401a-bfb1-6060520c875f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:50.661829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6d5de-592b-401a-bfb1-6060520c875f admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:50.662477 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6d5de-592b-401a-bfb1-6060520c875f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:50.662782 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6d5de-592b-401a-bfb1-6060520c875f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:50.664556 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6d5de-592b-401a-bfb1-6060520c875f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:50.676595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1063/2131] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:04:50.681258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd17447-14a1-4c63-91d6-89770ea2f4f7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:50.690689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd17447-14a1-4c63-91d6-89770ea2f4f7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:50.695524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa35f9-7566-4a72-b6b1-732baaaebd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:50.696159 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa35f9-7566-4a72-b6b1-732baaaebd84 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:50.696440 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa35f9-7566-4a72-b6b1-732baaaebd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:50.696714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aa35f9-7566-4a72-b6b1-732baaaebd84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:50.697350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa35f9-7566-4a72-b6b1-732baaaebd84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:50.699279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aa35f9-7566-4a72-b6b1-732baaaebd84 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:50.844393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6d5de-592b-401a-bfb1-6060520c875f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:50.847248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dde6d5de-592b-401a-bfb1-6060520c875f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:50.860621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1069/2132] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:50 2022] POST /identity/v3/projects => generated 397 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:50.861720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa35f9-7566-4a72-b6b1-732baaaebd84 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:50.921534 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6d5de-592b-401a-bfb1-6060520c875f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:50.947760 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aa35f9-7566-4a72-b6b1-732baaaebd84 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:50.953852 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffb91d-1988-408b-b7ba-5ed583f7b17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:50.954476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffb91d-1988-408b-b7ba-5ed583f7b17d admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:50.954830 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffb91d-1988-408b-b7ba-5ed583f7b17d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:50.955022 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ffb91d-1988-408b-b7ba-5ed583f7b17d admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:50.955682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffb91d-1988-408b-b7ba-5ed583f7b17d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:50.959101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ffb91d-1988-408b-b7ba-5ed583f7b17d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:51.000128 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78599c3e-2e9d-42cb-852d-bdf3d43a3d17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:51.000893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78599c3e-2e9d-42cb-852d-bdf3d43a3d17 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:51.001260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78599c3e-2e9d-42cb-852d-bdf3d43a3d17 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:51.001541 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78599c3e-2e9d-42cb-852d-bdf3d43a3d17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:51.004260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-78599c3e-2e9d-42cb-852d-bdf3d43a3d17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:51.050702 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1070/2133] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:50 2022] POST /identity/v3/users => generated 299 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:51.057029 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffb91d-1988-408b-b7ba-5ed583f7b17d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:51.068251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ffb91d-1988-408b-b7ba-5ed583f7b17d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:51.075071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb2725-9ec4-422c-b449-b35a7a135728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:51.076205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb2725-9ec4-422c-b449-b35a7a135728 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:51.076864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb2725-9ec4-422c-b449-b35a7a135728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:51.077183 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aeb2725-9ec4-422c-b449-b35a7a135728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:51.078328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb2725-9ec4-422c-b449-b35a7a135728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:51.081460 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aeb2725-9ec4-422c-b449-b35a7a135728 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:51.085425 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78599c3e-2e9d-42cb-852d-bdf3d43a3d17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:51.086281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78599c3e-2e9d-42cb-852d-bdf3d43a3d17 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:51.099415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1071/2134] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:51.099600 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1064/2135] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:04:51.100713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb2725-9ec4-422c-b449-b35a7a135728 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:51.111048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aeb2725-9ec4-422c-b449-b35a7a135728 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:51.117157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa5640-6823-4b6b-8454-f5b6b112e562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:51.117774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa5640-6823-4b6b-8454-f5b6b112e562 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:51.118122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa5640-6823-4b6b-8454-f5b6b112e562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:51.118461 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa5640-6823-4b6b-8454-f5b6b112e562 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:51.119116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa5640-6823-4b6b-8454-f5b6b112e562 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:51.120790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa5640-6823-4b6b-8454-f5b6b112e562 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:51.144072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1072/2136] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:51.148186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78599c3e-2e9d-42cb-852d-bdf3d43a3d17 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:51.161364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78599c3e-2e9d-42cb-852d-bdf3d43a3d17 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:51.166888 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e68e31e-0c4a-4d64-9faf-1e1c657fca1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:51.167734 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68e31e-0c4a-4d64-9faf-1e1c657fca1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:51.168023 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68e31e-0c4a-4d64-9faf-1e1c657fca1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:51.168402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e68e31e-0c4a-4d64-9faf-1e1c657fca1f admin admin] PATH_INFO: `/v3/projects/b3205249ee3b498cb112f347a4c1f7ce/users/3b460ad3d2214c6d8f25c16172a26bb7/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:51.197689 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68e31e-0c4a-4d64-9faf-1e1c657fca1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3205249ee3b498cb112f347a4c1f7ce, user_id=3b460ad3d2214c6d8f25c16172a26bb7, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:51.202809 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e68e31e-0c4a-4d64-9faf-1e1c657fca1f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:51.305201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1065/2137] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:51 2022] PUT /identity/v3/projects/b3205249ee3b498cb112f347a4c1f7ce/users/3b460ad3d2214c6d8f25c16172a26bb7/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:51.310020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa5640-6823-4b6b-8454-f5b6b112e562 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:51.315447 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-39fa5640-6823-4b6b-8454-f5b6b112e562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:51.320815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70573fe3-9702-4abf-ba3f-3386ebc22315 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:51.321360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70573fe3-9702-4abf-ba3f-3386ebc22315 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:51.321891 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70573fe3-9702-4abf-ba3f-3386ebc22315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:51.332705 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:51.333225 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa5640-6823-4b6b-8454-f5b6b112e562 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:51.376365 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-70573fe3-9702-4abf-ba3f-3386ebc22315 None None] MFA Rules not processed for user `3b460ad3d2214c6d8f25c16172a26bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:51.376365 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-70573fe3-9702-4abf-ba3f-3386ebc22315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:51.438754 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17d35b5-c7a3-4679-93b5-9649353cf329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:51.439804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d35b5-c7a3-4679-93b5-9649353cf329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:51.440269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d35b5-c7a3-4679-93b5-9649353cf329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:51.441327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17d35b5-c7a3-4679-93b5-9649353cf329 admin admin] PATH_INFO: `/v3/users/963df3106f3442c1ab0f141c24157632` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:51.448382 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d35b5-c7a3-4679-93b5-9649353cf329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=963df3106f3442c1ab0f141c24157632)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:51.448382 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17d35b5-c7a3-4679-93b5-9649353cf329 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:51.493629 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1066/2138] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:51 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:51.507746 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70573fe3-9702-4abf-ba3f-3386ebc22315 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:51.524265 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70573fe3-9702-4abf-ba3f-3386ebc22315 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:51.532208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157e959-846b-494e-90bf-103f7c3cecb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:51.532703 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157e959-846b-494e-90bf-103f7c3cecb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:51.533026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157e959-846b-494e-90bf-103f7c3cecb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:51.533388 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c157e959-846b-494e-90bf-103f7c3cecb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:51.534024 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157e959-846b-494e-90bf-103f7c3cecb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:51.536166 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c157e959-846b-494e-90bf-103f7c3cecb9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:51.737272 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1067/2139] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:51 2022] POST /identity/v3/projects => generated 401 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:51.746397 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157e959-846b-494e-90bf-103f7c3cecb9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:51.830895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c157e959-846b-494e-90bf-103f7c3cecb9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:51.836349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e35d00-7a6e-47ab-9d5d-1475bf909bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:51.836969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e35d00-7a6e-47ab-9d5d-1475bf909bfe admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:51.837261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e35d00-7a6e-47ab-9d5d-1475bf909bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:51.837633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e35d00-7a6e-47ab-9d5d-1475bf909bfe admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:51.838168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e35d00-7a6e-47ab-9d5d-1475bf909bfe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:51.840138 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e35d00-7a6e-47ab-9d5d-1475bf909bfe admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:51.891808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d17d35b5-c7a3-4679-93b5-9649353cf329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '963df3106f3442c1ab0f141c24157632'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:51.961580 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1068/2140] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:51 2022] POST /identity/v3/users => generated 307 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:51.969064 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e35d00-7a6e-47ab-9d5d-1475bf909bfe admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:51.994443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e35d00-7a6e-47ab-9d5d-1475bf909bfe admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:52.000125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9029bf-26a4-4713-85f4-eb11335dffa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:52.000911 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9029bf-26a4-4713-85f4-eb11335dffa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.001250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9029bf-26a4-4713-85f4-eb11335dffa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.001570 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9029bf-26a4-4713-85f4-eb11335dffa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.002338 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9029bf-26a4-4713-85f4-eb11335dffa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:52.004889 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9029bf-26a4-4713-85f4-eb11335dffa9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:52.018302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1069/2141] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:04:52.023214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9029bf-26a4-4713-85f4-eb11335dffa9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:52.034094 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9029bf-26a4-4713-85f4-eb11335dffa9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:52.039628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc593663-edcd-4187-854e-cbf906ef687a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:52.040344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc593663-edcd-4187-854e-cbf906ef687a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.040648 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc593663-edcd-4187-854e-cbf906ef687a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.040894 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc593663-edcd-4187-854e-cbf906ef687a admin admin] PATH_INFO: `/v3/projects/1d720749d3b740a9a28ddd3c71b3ff49/users/10a1d4c10ef04b5a81cbc687ec61be01/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.047728 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d17d35b5-c7a3-4679-93b5-9649353cf329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '963df3106f3442c1ab0f141c24157632'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:52.048637 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d17d35b5-c7a3-4679-93b5-9649353cf329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '963df3106f3442c1ab0f141c24157632'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:52.049285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d17d35b5-c7a3-4679-93b5-9649353cf329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '963df3106f3442c1ab0f141c24157632'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:52.061267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d17d35b5-c7a3-4679-93b5-9649353cf329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '963df3106f3442c1ab0f141c24157632'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:52.061574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc593663-edcd-4187-854e-cbf906ef687a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d720749d3b740a9a28ddd3c71b3ff49, user_id=10a1d4c10ef04b5a81cbc687ec61be01, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:52.064308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc593663-edcd-4187-854e-cbf906ef687a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:52.088438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1073/2142] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:51 2022] DELETE /identity/v3/users/963df3106f3442c1ab0f141c24157632 => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:52.092827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e0a0d-dcf9-4c6f-8105-c507be66c99f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.093137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e0a0d-dcf9-4c6f-8105-c507be66c99f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.093441 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2e0a0d-dcf9-4c6f-8105-c507be66c99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.120060 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2e2e0a0d-dcf9-4c6f-8105-c507be66c99f None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:52.122034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2e2e0a0d-dcf9-4c6f-8105-c507be66c99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:52.195550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1074/2143] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:52.196249 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e0a0d-dcf9-4c6f-8105-c507be66c99f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:52.197613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2e2e0a0d-dcf9-4c6f-8105-c507be66c99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:52.212488 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2e0a0d-dcf9-4c6f-8105-c507be66c99f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:52.217790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbace76-bd22-4830-8660-e89eb5771048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:52.218426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbace76-bd22-4830-8660-e89eb5771048 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.218699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbace76-bd22-4830-8660-e89eb5771048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.219071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbace76-bd22-4830-8660-e89eb5771048 admin admin] PATH_INFO: `/v3/users/198fac9935ba44cfaaa7a17835f5aae3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.220212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbace76-bd22-4830-8660-e89eb5771048 admin admin] RBAC: Authorizing `identity:delete_user(user_id=198fac9935ba44cfaaa7a17835f5aae3)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:52.222481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbace76-bd22-4830-8660-e89eb5771048 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:52.251154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1070/2144] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:52 2022] PUT /identity/v3/projects/1d720749d3b740a9a28ddd3c71b3ff49/users/10a1d4c10ef04b5a81cbc687ec61be01/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:52.252304 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc593663-edcd-4187-854e-cbf906ef687a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:52.255000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bc593663-edcd-4187-854e-cbf906ef687a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:52.318881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc593663-edcd-4187-854e-cbf906ef687a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:52.322976 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad6da0-9bcc-471e-92f6-4ced46d20f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:52.323626 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad6da0-9bcc-471e-92f6-4ced46d20f57 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.324019 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad6da0-9bcc-471e-92f6-4ced46d20f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.324332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad6da0-9bcc-471e-92f6-4ced46d20f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.401262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad6da0-9bcc-471e-92f6-4ced46d20f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:52.403851 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ad6da0-9bcc-471e-92f6-4ced46d20f57 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:52.427738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1071/2145] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:52.430072 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad6da0-9bcc-471e-92f6-4ced46d20f57 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:52.433803 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f8ad6da0-9bcc-471e-92f6-4ced46d20f57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:52.462628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad6da0-9bcc-471e-92f6-4ced46d20f57 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:52.469271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2f186-a38a-4c8c-bc19-b1e12b3dcccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:52.469981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2f186-a38a-4c8c-bc19-b1e12b3dcccd admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.470400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2f186-a38a-4c8c-bc19-b1e12b3dcccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.470850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff2f186-a38a-4c8c-bc19-b1e12b3dcccd admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.471705 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2f186-a38a-4c8c-bc19-b1e12b3dcccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:52.473646 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff2f186-a38a-4c8c-bc19-b1e12b3dcccd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:52.493586 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1072/2146] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:52.497889 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2f186-a38a-4c8c-bc19-b1e12b3dcccd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:52.509542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff2f186-a38a-4c8c-bc19-b1e12b3dcccd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:52.514598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863f9c9-9a1c-4efe-8ac0-b8f3c6d4b868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:52.515264 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863f9c9-9a1c-4efe-8ac0-b8f3c6d4b868 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.515699 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863f9c9-9a1c-4efe-8ac0-b8f3c6d4b868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.516070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863f9c9-9a1c-4efe-8ac0-b8f3c6d4b868 admin admin] PATH_INFO: `/v3/projects/1d720749d3b740a9a28ddd3c71b3ff49/users/10a1d4c10ef04b5a81cbc687ec61be01/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.518209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863f9c9-9a1c-4efe-8ac0-b8f3c6d4b868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d720749d3b740a9a28ddd3c71b3ff49, user_id=10a1d4c10ef04b5a81cbc687ec61be01, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:52.521045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863f9c9-9a1c-4efe-8ac0-b8f3c6d4b868 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:52.533096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1073/2147] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:52 2022] PUT /identity/v3/projects/1d720749d3b740a9a28ddd3c71b3ff49/users/10a1d4c10ef04b5a81cbc687ec61be01/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:52.549354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac293a9-53f5-4dc4-ba64-8bce3f89b698 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.549643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac293a9-53f5-4dc4-ba64-8bce3f89b698 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.550124 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac293a9-53f5-4dc4-ba64-8bce3f89b698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.595096 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ffbace76-bd22-4830-8660-e89eb5771048 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '198fac9935ba44cfaaa7a17835f5aae3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:52.599155 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8ac293a9-53f5-4dc4-ba64-8bce3f89b698 None None] MFA Rules not processed for user `10a1d4c10ef04b5a81cbc687ec61be01`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:52.600550 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8ac293a9-53f5-4dc4-ba64-8bce3f89b698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:52.695153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1074/2148] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:52 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:52.706517 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac293a9-53f5-4dc4-ba64-8bce3f89b698 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:52.708911 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8ac293a9-53f5-4dc4-ba64-8bce3f89b698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:52.754403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ffbace76-bd22-4830-8660-e89eb5771048 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '198fac9935ba44cfaaa7a17835f5aae3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:52.755162 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ffbace76-bd22-4830-8660-e89eb5771048 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '198fac9935ba44cfaaa7a17835f5aae3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:52.755894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ffbace76-bd22-4830-8660-e89eb5771048 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '198fac9935ba44cfaaa7a17835f5aae3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:52.764625 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ffbace76-bd22-4830-8660-e89eb5771048 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '198fac9935ba44cfaaa7a17835f5aae3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:52.785935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac293a9-53f5-4dc4-ba64-8bce3f89b698 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:52.789858 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790b872-fac3-4253-a6e1-80c000db4143 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:52.791345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790b872-fac3-4253-a6e1-80c000db4143 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.791345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790b872-fac3-4253-a6e1-80c000db4143 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.791345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7790b872-fac3-4253-a6e1-80c000db4143 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.795570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1075/2149] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:52 2022] DELETE /identity/v3/users/198fac9935ba44cfaaa7a17835f5aae3 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:52.796719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7790b872-fac3-4253-a6e1-80c000db4143 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:52.801612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56415eb-938f-4e58-8bea-dab71d2a66b4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:52.801612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56415eb-938f-4e58-8bea-dab71d2a66b4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:52.801612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56415eb-938f-4e58-8bea-dab71d2a66b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:52.818981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790b872-fac3-4253-a6e1-80c000db4143 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:52.821659 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7790b872-fac3-4253-a6e1-80c000db4143 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:52.826619 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a56415eb-938f-4e58-8bea-dab71d2a66b4 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:52.828028 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a56415eb-938f-4e58-8bea-dab71d2a66b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:52.839551 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1075/2150] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:52 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:04:52.911786 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1076/2151] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:52.918605 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790b872-fac3-4253-a6e1-80c000db4143 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:52.920650 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7790b872-fac3-4253-a6e1-80c000db4143 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:53.050296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7790b872-fac3-4253-a6e1-80c000db4143 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:53.118016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3a295-d43d-4e40-8069-0964ad6124a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:53.118636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3a295-d43d-4e40-8069-0964ad6124a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:53.118914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3a295-d43d-4e40-8069-0964ad6124a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:53.119179 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a3a295-d43d-4e40-8069-0964ad6124a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:53.131840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3a295-d43d-4e40-8069-0964ad6124a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:53.133307 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a3a295-d43d-4e40-8069-0964ad6124a1 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:53.144407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1076/2152] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:53.219788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56415eb-938f-4e58-8bea-dab71d2a66b4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:53.221528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a56415eb-938f-4e58-8bea-dab71d2a66b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:53.240185 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56415eb-938f-4e58-8bea-dab71d2a66b4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:53.246481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5f1a7-78cf-48a1-92ce-00f50017ec32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:53.247133 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5f1a7-78cf-48a1-92ce-00f50017ec32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:53.247409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5f1a7-78cf-48a1-92ce-00f50017ec32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:53.247915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5f1a7-78cf-48a1-92ce-00f50017ec32 admin admin] PATH_INFO: `/v3/projects/b7dc54fef855410193af3b417a32abc8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:53.249072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5f1a7-78cf-48a1-92ce-00f50017ec32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7dc54fef855410193af3b417a32abc8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:53.250979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5f1a7-78cf-48a1-92ce-00f50017ec32 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:53.439859 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-33d5f1a7-78cf-48a1-92ce-00f50017ec32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7dc54fef855410193af3b417a32abc8', 'request_id': 'req-33d5f1a7-78cf-48a1-92ce-00f50017ec32'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:53.449930 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3a295-d43d-4e40-8069-0964ad6124a1 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:53.450217 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d8a3a295-d43d-4e40-8069-0964ad6124a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:53.453369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-33d5f1a7-78cf-48a1-92ce-00f50017ec32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7dc54fef855410193af3b417a32abc8', 'request_id': 'req-33d5f1a7-78cf-48a1-92ce-00f50017ec32'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:53.537364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a3a295-d43d-4e40-8069-0964ad6124a1 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:53.629920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-33d5f1a7-78cf-48a1-92ce-00f50017ec32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7dc54fef855410193af3b417a32abc8', 'request_id': 'req-33d5f1a7-78cf-48a1-92ce-00f50017ec32'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:53.636118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c57af-2172-4ad1-9336-b5a9686ca118 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:53.639064 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c57af-2172-4ad1-9336-b5a9686ca118 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:53.639064 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c57af-2172-4ad1-9336-b5a9686ca118 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:53.639064 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48c57af-2172-4ad1-9336-b5a9686ca118 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:53.643452 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e48c57af-2172-4ad1-9336-b5a9686ca118 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:53.649587 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-33d5f1a7-78cf-48a1-92ce-00f50017ec32 admin admin] The token cache is being invalidate because project b7dc54fef855410193af3b417a32abc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:53.650025 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-33d5f1a7-78cf-48a1-92ce-00f50017ec32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:53.654252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-33d5f1a7-78cf-48a1-92ce-00f50017ec32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:53.656317 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1077/2153] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:53 2022] DELETE /identity/v3/projects/b7dc54fef855410193af3b417a32abc8 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:53.661576 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5f1a7-78cf-48a1-92ce-00f50017ec32 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:53.670476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-33d5f1a7-78cf-48a1-92ce-00f50017ec32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:53.785219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:53.794687 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c57af-2172-4ad1-9336-b5a9686ca118 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:53.798903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48c57af-2172-4ad1-9336-b5a9686ca118 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:53.820121 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1077/2154] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:53 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:04:53.822421 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5f1a7-78cf-48a1-92ce-00f50017ec32 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:53.832144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d389b-4341-436a-bdf4-0f429a339c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:53.832144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d389b-4341-436a-bdf4-0f429a339c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:53.832144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d389b-4341-436a-bdf4-0f429a339c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:53.832144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045d389b-4341-436a-bdf4-0f429a339c8b admin admin] PATH_INFO: `/v3/users/6780f39a47e244c0824f880e88d3a235` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:53.833609 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d389b-4341-436a-bdf4-0f429a339c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6780f39a47e244c0824f880e88d3a235)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:53.835951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045d389b-4341-436a-bdf4-0f429a339c8b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:53.906774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c57af-2172-4ad1-9336-b5a9686ca118 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:53.906774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e48c57af-2172-4ad1-9336-b5a9686ca118 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:53.928290 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48c57af-2172-4ad1-9336-b5a9686ca118 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:53.935832 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f1f68-60eb-444b-8e35-445370da98e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:53.935995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:53.936723 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f1f68-60eb-444b-8e35-445370da98e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:53.937095 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f1f68-60eb-444b-8e35-445370da98e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:53.937522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f1f68-60eb-444b-8e35-445370da98e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:53.940371 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ff6f1f68-60eb-444b-8e35-445370da98e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:54.074474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f1f68-60eb-444b-8e35-445370da98e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:54.075957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f1f68-60eb-444b-8e35-445370da98e3 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:54.086701 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1078/2155] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:54.087523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f1f68-60eb-444b-8e35-445370da98e3 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:54.089036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ff6f1f68-60eb-444b-8e35-445370da98e3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:54.106292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f1f68-60eb-444b-8e35-445370da98e3 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:54.114383 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16746131-0fa2-4ef9-b6e0-0755180794c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:54.115586 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16746131-0fa2-4ef9-b6e0-0755180794c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:54.116194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16746131-0fa2-4ef9-b6e0-0755180794c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:54.116653 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16746131-0fa2-4ef9-b6e0-0755180794c5 admin admin] PATH_INFO: `/v3/projects/4f285741e21a4c5a904058e941059e1f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:54.118590 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16746131-0fa2-4ef9-b6e0-0755180794c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f285741e21a4c5a904058e941059e1f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:54.122391 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16746131-0fa2-4ef9-b6e0-0755180794c5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:54.231688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-045d389b-4341-436a-bdf4-0f429a339c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6780f39a47e244c0824f880e88d3a235'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:54.304533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-16746131-0fa2-4ef9-b6e0-0755180794c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f285741e21a4c5a904058e941059e1f', 'request_id': 'req-16746131-0fa2-4ef9-b6e0-0755180794c5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:54.311905 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-16746131-0fa2-4ef9-b6e0-0755180794c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f285741e21a4c5a904058e941059e1f', 'request_id': 'req-16746131-0fa2-4ef9-b6e0-0755180794c5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:54.432845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-045d389b-4341-436a-bdf4-0f429a339c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6780f39a47e244c0824f880e88d3a235'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:54.433876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-045d389b-4341-436a-bdf4-0f429a339c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6780f39a47e244c0824f880e88d3a235'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:54.434972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-045d389b-4341-436a-bdf4-0f429a339c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6780f39a47e244c0824f880e88d3a235'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:54.446347 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-045d389b-4341-436a-bdf4-0f429a339c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6780f39a47e244c0824f880e88d3a235'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:54.479975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1078/2156] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:53 2022] DELETE /identity/v3/users/6780f39a47e244c0824f880e88d3a235 => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:54.626701 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-16746131-0fa2-4ef9-b6e0-0755180794c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f285741e21a4c5a904058e941059e1f', 'request_id': 'req-16746131-0fa2-4ef9-b6e0-0755180794c5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:54.639314 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-16746131-0fa2-4ef9-b6e0-0755180794c5 admin admin] The token cache is being invalidate because project 4f285741e21a4c5a904058e941059e1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:54.639611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-16746131-0fa2-4ef9-b6e0-0755180794c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:54.640290 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-16746131-0fa2-4ef9-b6e0-0755180794c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:54.642018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1079/2157] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:54 2022] DELETE /identity/v3/projects/4f285741e21a4c5a904058e941059e1f => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:54.646916 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d389b-4341-436a-bdf4-0f429a339c8b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:54.648612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-045d389b-4341-436a-bdf4-0f429a339c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:54.745752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045d389b-4341-436a-bdf4-0f429a339c8b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:54.770387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cfbb8-9d4c-49ff-84b8-f094dba5dac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:54.770387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cfbb8-9d4c-49ff-84b8-f094dba5dac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:54.770387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cfbb8-9d4c-49ff-84b8-f094dba5dac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:54.770387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3cfbb8-9d4c-49ff-84b8-f094dba5dac0 admin admin] PATH_INFO: `/v3/users/b06aba66339d47b1be1876a222e9cf2d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:54.770734 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cfbb8-9d4c-49ff-84b8-f094dba5dac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b06aba66339d47b1be1876a222e9cf2d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:54.773991 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3cfbb8-9d4c-49ff-84b8-f094dba5dac0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:55.161179 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16746131-0fa2-4ef9-b6e0-0755180794c5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:55.163026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-16746131-0fa2-4ef9-b6e0-0755180794c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:55.233693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1a3cfbb8-9d4c-49ff-84b8-f094dba5dac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b06aba66339d47b1be1876a222e9cf2d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:55.310581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16746131-0fa2-4ef9-b6e0-0755180794c5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:55.319377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf57e1-5e19-46f9-a0b8-97e563371f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:55.320515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf57e1-5e19-46f9-a0b8-97e563371f3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:55.321118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf57e1-5e19-46f9-a0b8-97e563371f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:55.322054 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bf57e1-5e19-46f9-a0b8-97e563371f3f admin admin] PATH_INFO: `/v3/projects/3adca5d2175f4856b8551614813e6612` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:55.323342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf57e1-5e19-46f9-a0b8-97e563371f3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3adca5d2175f4856b8551614813e6612)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:55.327201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bf57e1-5e19-46f9-a0b8-97e563371f3f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:55.387774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1a3cfbb8-9d4c-49ff-84b8-f094dba5dac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b06aba66339d47b1be1876a222e9cf2d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:55.388778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1a3cfbb8-9d4c-49ff-84b8-f094dba5dac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b06aba66339d47b1be1876a222e9cf2d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:55.389613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1a3cfbb8-9d4c-49ff-84b8-f094dba5dac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b06aba66339d47b1be1876a222e9cf2d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:55.400293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1a3cfbb8-9d4c-49ff-84b8-f094dba5dac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b06aba66339d47b1be1876a222e9cf2d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:55.428989 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1079/2158] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:54 2022] DELETE /identity/v3/users/b06aba66339d47b1be1876a222e9cf2d => generated 0 bytes in 783 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:55.919320 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-81bf57e1-5e19-46f9-a0b8-97e563371f3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3adca5d2175f4856b8551614813e6612', 'request_id': 'req-81bf57e1-5e19-46f9-a0b8-97e563371f3f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:55.931383 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-81bf57e1-5e19-46f9-a0b8-97e563371f3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3adca5d2175f4856b8551614813e6612', 'request_id': 'req-81bf57e1-5e19-46f9-a0b8-97e563371f3f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:56.101179 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-81bf57e1-5e19-46f9-a0b8-97e563371f3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3adca5d2175f4856b8551614813e6612', 'request_id': 'req-81bf57e1-5e19-46f9-a0b8-97e563371f3f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:56.115641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-81bf57e1-5e19-46f9-a0b8-97e563371f3f admin admin] The token cache is being invalidate because project 3adca5d2175f4856b8551614813e6612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:56.119195 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-81bf57e1-5e19-46f9-a0b8-97e563371f3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:56.119195 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-81bf57e1-5e19-46f9-a0b8-97e563371f3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:56.119195 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1080/2159] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:55 2022] DELETE /identity/v3/projects/3adca5d2175f4856b8551614813e6612 => generated 0 bytes in 959 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:56.123369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cfbb8-9d4c-49ff-84b8-f094dba5dac0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:56.131748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1a3cfbb8-9d4c-49ff-84b8-f094dba5dac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:56.181007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf57e1-5e19-46f9-a0b8-97e563371f3f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:56.183037 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-81bf57e1-5e19-46f9-a0b8-97e563371f3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:56.242379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3cfbb8-9d4c-49ff-84b8-f094dba5dac0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:56.276406 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bf57e1-5e19-46f9-a0b8-97e563371f3f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:56.283886 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b9d94-5298-4f61-beba-e3fb7b8c5064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:56.285036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b9d94-5298-4f61-beba-e3fb7b8c5064 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:56.285446 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b9d94-5298-4f61-beba-e3fb7b8c5064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:56.285850 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b9d94-5298-4f61-beba-e3fb7b8c5064 admin admin] PATH_INFO: `/v3/projects/b987abb21feb4024888c0353450fb058` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:56.288854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b9d94-5298-4f61-beba-e3fb7b8c5064 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b987abb21feb4024888c0353450fb058)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:56.292282 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b9d94-5298-4f61-beba-e3fb7b8c5064 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:56.324836 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fbf69-c077-4329-8701-5f79cca10be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:56.326049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fbf69-c077-4329-8701-5f79cca10be1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:56.326812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fbf69-c077-4329-8701-5f79cca10be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:56.327362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fbf69-c077-4329-8701-5f79cca10be1 admin admin] PATH_INFO: `/v3/users/0eb1ba23d4004a20b7220c8471a9382c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:56.329554 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fbf69-c077-4329-8701-5f79cca10be1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0eb1ba23d4004a20b7220c8471a9382c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:56.333759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fbf69-c077-4329-8701-5f79cca10be1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:56.577442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-eb1b9d94-5298-4f61-beba-e3fb7b8c5064 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b987abb21feb4024888c0353450fb058', 'request_id': 'req-eb1b9d94-5298-4f61-beba-e3fb7b8c5064'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:56.585386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-eb1b9d94-5298-4f61-beba-e3fb7b8c5064 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b987abb21feb4024888c0353450fb058', 'request_id': 'req-eb1b9d94-5298-4f61-beba-e3fb7b8c5064'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:56.831619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-eb1b9d94-5298-4f61-beba-e3fb7b8c5064 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b987abb21feb4024888c0353450fb058', 'request_id': 'req-eb1b9d94-5298-4f61-beba-e3fb7b8c5064'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:56.844868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-eb1b9d94-5298-4f61-beba-e3fb7b8c5064 admin admin] The token cache is being invalidate because project b987abb21feb4024888c0353450fb058 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:56.845161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-eb1b9d94-5298-4f61-beba-e3fb7b8c5064 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:56.845843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-eb1b9d94-5298-4f61-beba-e3fb7b8c5064 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:56.847969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1081/2160] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:56 2022] DELETE /identity/v3/projects/b987abb21feb4024888c0353450fb058 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:56.853778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e77a6-c958-4e51-b826-3c6c5f27dc79 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:56.854070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e77a6-c958-4e51-b826-3c6c5f27dc79 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:56.854458 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70e77a6-c958-4e51-b826-3c6c5f27dc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:56.855384 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-490fbf69-c077-4329-8701-5f79cca10be1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0eb1ba23d4004a20b7220c8471a9382c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:56.877405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e70e77a6-c958-4e51-b826-3c6c5f27dc79 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:56.879463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e70e77a6-c958-4e51-b826-3c6c5f27dc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:56.958348 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1082/2161] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:56.969513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea56257-3b3f-420b-b271-9ba50eec48ed None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:56.970113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea56257-3b3f-420b-b271-9ba50eec48ed None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:56.970678 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea56257-3b3f-420b-b271-9ba50eec48ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:56.987793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-490fbf69-c077-4329-8701-5f79cca10be1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0eb1ba23d4004a20b7220c8471a9382c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:56.989360 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-490fbf69-c077-4329-8701-5f79cca10be1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0eb1ba23d4004a20b7220c8471a9382c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:56.990864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-490fbf69-c077-4329-8701-5f79cca10be1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0eb1ba23d4004a20b7220c8471a9382c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:57.011181 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bea56257-3b3f-420b-b271-9ba50eec48ed None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:57.012614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bea56257-3b3f-420b-b271-9ba50eec48ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:57.027085 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-490fbf69-c077-4329-8701-5f79cca10be1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0eb1ba23d4004a20b7220c8471a9382c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:57.029048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1083/2162] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:57.036552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea56257-3b3f-420b-b271-9ba50eec48ed None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:57.053619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea56257-3b3f-420b-b271-9ba50eec48ed None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:57.069300 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d6d92-dff8-4d13-9dcb-a075f5b3536d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:57.070309 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d6d92-dff8-4d13-9dcb-a075f5b3536d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:57.070772 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d6d92-dff8-4d13-9dcb-a075f5b3536d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:57.071229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d6d92-dff8-4d13-9dcb-a075f5b3536d admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:57.072647 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d6d92-dff8-4d13-9dcb-a075f5b3536d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:57.073497 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d6d92-dff8-4d13-9dcb-a075f5b3536d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:57.077249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308d6d92-dff8-4d13-9dcb-a075f5b3536d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:57.090711 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1080/2163] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:56 2022] DELETE /identity/v3/users/0eb1ba23d4004a20b7220c8471a9382c => generated 0 bytes in 961 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:57.110623 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1084/2164] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:04:57.112181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fbf69-c077-4329-8701-5f79cca10be1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:57.238843 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fbf69-c077-4329-8701-5f79cca10be1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:57.247483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78947ae2-dcf1-48c3-ab21-de3cd0a35982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:57.248652 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78947ae2-dcf1-48c3-ab21-de3cd0a35982 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:57.249246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78947ae2-dcf1-48c3-ab21-de3cd0a35982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:57.249917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78947ae2-dcf1-48c3-ab21-de3cd0a35982 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:57.251274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78947ae2-dcf1-48c3-ab21-de3cd0a35982 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:57.255372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78947ae2-dcf1-48c3-ab21-de3cd0a35982 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:57.262941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:57.364709 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=78,keystone:DELETE=19,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:04:57.379112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1081/2165] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:57 2022] POST /identity/v3/projects => generated 355 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:57.385456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d6d92-dff8-4d13-9dcb-a075f5b3536d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:57.409469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d6d92-dff8-4d13-9dcb-a075f5b3536d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:57.451280 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78947ae2-dcf1-48c3-ab21-de3cd0a35982 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:57.455783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-78947ae2-dcf1-48c3-ab21-de3cd0a35982 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:57.535435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa340c8-c9b8-4dfd-a5c3-0b412b106ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:57.537038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa340c8-c9b8-4dfd-a5c3-0b412b106ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:57.537368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa340c8-c9b8-4dfd-a5c3-0b412b106ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:57.537616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa340c8-c9b8-4dfd-a5c3-0b412b106ff0 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:57.538701 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa340c8-c9b8-4dfd-a5c3-0b412b106ff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:57.541798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa340c8-c9b8-4dfd-a5c3-0b412b106ff0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:57.596056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78947ae2-dcf1-48c3-ab21-de3cd0a35982 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:57.603615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace8b82-5f60-48ba-9931-4fc7110bb359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:57.603615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace8b82-5f60-48ba-9931-4fc7110bb359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:57.604291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace8b82-5f60-48ba-9931-4fc7110bb359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:57.604792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace8b82-5f60-48ba-9931-4fc7110bb359 admin admin] PATH_INFO: `/v3/users/e71afa4b7ff24c86aacdeb2b4a3d73f6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:57.606672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace8b82-5f60-48ba-9931-4fc7110bb359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e71afa4b7ff24c86aacdeb2b4a3d73f6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:57.609838 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ace8b82-5f60-48ba-9931-4fc7110bb359 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:57.778902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1085/2166] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:57 2022] POST /identity/v3/users => generated 278 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:57.788931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa340c8-c9b8-4dfd-a5c3-0b412b106ff0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:57.800216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa340c8-c9b8-4dfd-a5c3-0b412b106ff0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:57.826585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c5706-62a1-435e-bcd0-ad7cf0102e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:57.827823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c5706-62a1-435e-bcd0-ad7cf0102e6c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:57.828508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c5706-62a1-435e-bcd0-ad7cf0102e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:57.829066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c5706-62a1-435e-bcd0-ad7cf0102e6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:57.830371 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c5706-62a1-435e-bcd0-ad7cf0102e6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:57.841980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556c5706-62a1-435e-bcd0-ad7cf0102e6c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:57.867874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1086/2167] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:57.867874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c5706-62a1-435e-bcd0-ad7cf0102e6c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:57.867874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-556c5706-62a1-435e-bcd0-ad7cf0102e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:57.884072 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c5706-62a1-435e-bcd0-ad7cf0102e6c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:57.889245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a0b058-9140-4359-8b01-12dbe6ef64d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:57.889885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0b058-9140-4359-8b01-12dbe6ef64d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:57.890171 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0b058-9140-4359-8b01-12dbe6ef64d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:57.890446 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a0b058-9140-4359-8b01-12dbe6ef64d5 admin admin] PATH_INFO: `/v3/projects/d044ffb98bf94c55994753650de92de5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:57.891578 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0b058-9140-4359-8b01-12dbe6ef64d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d044ffb98bf94c55994753650de92de5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:57.893546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a0b058-9140-4359-8b01-12dbe6ef64d5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:58.061471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6ace8b82-5f60-48ba-9931-4fc7110bb359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e71afa4b7ff24c86aacdeb2b4a3d73f6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:58.138190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-74a0b058-9140-4359-8b01-12dbe6ef64d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd044ffb98bf94c55994753650de92de5', 'request_id': 'req-74a0b058-9140-4359-8b01-12dbe6ef64d5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:58.148863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-74a0b058-9140-4359-8b01-12dbe6ef64d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd044ffb98bf94c55994753650de92de5', 'request_id': 'req-74a0b058-9140-4359-8b01-12dbe6ef64d5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:58.181139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6ace8b82-5f60-48ba-9931-4fc7110bb359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e71afa4b7ff24c86aacdeb2b4a3d73f6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:58.182571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6ace8b82-5f60-48ba-9931-4fc7110bb359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e71afa4b7ff24c86aacdeb2b4a3d73f6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:58.183366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6ace8b82-5f60-48ba-9931-4fc7110bb359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e71afa4b7ff24c86aacdeb2b4a3d73f6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:58.194457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6ace8b82-5f60-48ba-9931-4fc7110bb359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e71afa4b7ff24c86aacdeb2b4a3d73f6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:58.224768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1082/2168] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:04:57 2022] DELETE /identity/v3/users/e71afa4b7ff24c86aacdeb2b4a3d73f6 => generated 0 bytes in 774 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:58.233478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace8b82-5f60-48ba-9931-4fc7110bb359 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:58.233478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6ace8b82-5f60-48ba-9931-4fc7110bb359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:58.323810 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-74a0b058-9140-4359-8b01-12dbe6ef64d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd044ffb98bf94c55994753650de92de5', 'request_id': 'req-74a0b058-9140-4359-8b01-12dbe6ef64d5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:58.326237 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace8b82-5f60-48ba-9931-4fc7110bb359 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:58.332492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bdcbf-3ded-4945-a367-278fe5c09360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:58.333199 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bdcbf-3ded-4945-a367-278fe5c09360 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:58.333483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bdcbf-3ded-4945-a367-278fe5c09360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:58.333756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71bdcbf-3ded-4945-a367-278fe5c09360 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:58.334412 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71bdcbf-3ded-4945-a367-278fe5c09360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:58.336071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71bdcbf-3ded-4945-a367-278fe5c09360 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:58.343759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-74a0b058-9140-4359-8b01-12dbe6ef64d5 admin admin] The token cache is being invalidate because project d044ffb98bf94c55994753650de92de5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:04:58.344227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-74a0b058-9140-4359-8b01-12dbe6ef64d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:58.344936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-74a0b058-9140-4359-8b01-12dbe6ef64d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:58.348407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1087/2169] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:57 2022] DELETE /identity/v3/projects/d044ffb98bf94c55994753650de92de5 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:04:58.350116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1083/2170] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:58 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:58.350254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449f17b-35db-4eb6-82ed-4db86e95039b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:58.350254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449f17b-35db-4eb6-82ed-4db86e95039b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:58.350254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449f17b-35db-4eb6-82ed-4db86e95039b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:58.354127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bdcbf-3ded-4945-a367-278fe5c09360 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:58.356312 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e71bdcbf-3ded-4945-a367-278fe5c09360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:58.369361 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71bdcbf-3ded-4945-a367-278fe5c09360 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:58.374634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932d3944-9316-4f85-89c7-a829f2842ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:58.375304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d3944-9316-4f85-89c7-a829f2842ace admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:58.375573 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d3944-9316-4f85-89c7-a829f2842ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:58.375988 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932d3944-9316-4f85-89c7-a829f2842ace admin admin] PATH_INFO: `/v3/projects/f01b46a2d7c94d24b22baffd180902a4/users/a50100444983412180bb2b34bfefd7e2/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:58.384481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1449f17b-35db-4eb6-82ed-4db86e95039b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:58.386032 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1449f17b-35db-4eb6-82ed-4db86e95039b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:58.394580 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1088/2171] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:58.395833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b43f46-5dc0-44ed-adb1-b02c56bab500 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:58.396119 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b43f46-5dc0-44ed-adb1-b02c56bab500 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:58.396392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b43f46-5dc0-44ed-adb1-b02c56bab500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:58.400964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d3944-9316-4f85-89c7-a829f2842ace admin admin] RBAC: Authorizing `identity:create_grant(project_id=f01b46a2d7c94d24b22baffd180902a4, user_id=a50100444983412180bb2b34bfefd7e2, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:58.403730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932d3944-9316-4f85-89c7-a829f2842ace admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:58.423327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-86b43f46-5dc0-44ed-adb1-b02c56bab500 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:58.424704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-86b43f46-5dc0-44ed-adb1-b02c56bab500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:58.433414 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1089/2172] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:58.434298 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b43f46-5dc0-44ed-adb1-b02c56bab500 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:58.435604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-86b43f46-5dc0-44ed-adb1-b02c56bab500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:58.503100 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b43f46-5dc0-44ed-adb1-b02c56bab500 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:58.507126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31552ef-7da2-4da9-96b3-455beba9627e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:58.508172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31552ef-7da2-4da9-96b3-455beba9627e service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:58.508617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31552ef-7da2-4da9-96b3-455beba9627e service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:58.509062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31552ef-7da2-4da9-96b3-455beba9627e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:58.526347 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31552ef-7da2-4da9-96b3-455beba9627e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:58.528902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31552ef-7da2-4da9-96b3-455beba9627e service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:58.546490 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1090/2173] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:04:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:04:58.548553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc7f00-46a4-40e0-8b05-40d113581e82 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:58.549008 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc7f00-46a4-40e0-8b05-40d113581e82 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:58.549441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdc7f00-46a4-40e0-8b05-40d113581e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:58.581636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7bdc7f00-46a4-40e0-8b05-40d113581e82 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:58.584642 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7bdc7f00-46a4-40e0-8b05-40d113581e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:58.595441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1091/2174] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:04:58.599952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc7f00-46a4-40e0-8b05-40d113581e82 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:58.612777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdc7f00-46a4-40e0-8b05-40d113581e82 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:58.619787 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd5a3b-6ff9-41f5-8baf-a9d02cb151ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:58.620429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd5a3b-6ff9-41f5-8baf-a9d02cb151ef admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:58.620725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd5a3b-6ff9-41f5-8baf-a9d02cb151ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:58.621118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd5a3b-6ff9-41f5-8baf-a9d02cb151ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:58.622259 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd5a3b-6ff9-41f5-8baf-a9d02cb151ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:04:58.622612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd5a3b-6ff9-41f5-8baf-a9d02cb151ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:58.625688 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd5a3b-6ff9-41f5-8baf-a9d02cb151ef admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:58.648493 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1092/2175] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:04:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:04:58.653692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd5a3b-6ff9-41f5-8baf-a9d02cb151ef admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:58.665368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd5a3b-6ff9-41f5-8baf-a9d02cb151ef admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:58.670606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf0a63-f111-481d-bb09-533d6b09f186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:58.692432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf0a63-f111-481d-bb09-533d6b09f186 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:58.692432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf0a63-f111-481d-bb09-533d6b09f186 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:58.692432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf0a63-f111-481d-bb09-533d6b09f186 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:58.692432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf0a63-f111-481d-bb09-533d6b09f186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:58.692432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf0a63-f111-481d-bb09-533d6b09f186 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:58.765151 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1084/2176] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:58 2022] PUT /identity/v3/projects/f01b46a2d7c94d24b22baffd180902a4/users/a50100444983412180bb2b34bfefd7e2/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:04:58.775974 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7092ee-334a-4dfe-b1e8-42a9955f6aea None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:58.776254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7092ee-334a-4dfe-b1e8-42a9955f6aea None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:58.776524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7092ee-334a-4dfe-b1e8-42a9955f6aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:58.806802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1d7092ee-334a-4dfe-b1e8-42a9955f6aea None None] MFA Rules not processed for user `a50100444983412180bb2b34bfefd7e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:04:58.808577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1d7092ee-334a-4dfe-b1e8-42a9955f6aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:58.915208 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1085/2177] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:04:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:04:58.920435 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7092ee-334a-4dfe-b1e8-42a9955f6aea None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:59.011293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7092ee-334a-4dfe-b1e8-42a9955f6aea None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:59.020418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97717437-a29a-4231-9abc-25af0e64e400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:59.021436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97717437-a29a-4231-9abc-25af0e64e400 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:59.021923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97717437-a29a-4231-9abc-25af0e64e400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:59.022403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97717437-a29a-4231-9abc-25af0e64e400 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:59.023520 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97717437-a29a-4231-9abc-25af0e64e400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:59.026896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97717437-a29a-4231-9abc-25af0e64e400 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:59.102462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1093/2178] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:58 2022] POST /identity/v3/projects => generated 371 bytes in 441 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:04:59.113721 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf0a63-f111-481d-bb09-533d6b09f186 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:59.237725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf0a63-f111-481d-bb09-533d6b09f186 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:59.253646 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f82a8-e80c-4949-a2c4-e08f516c06a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:59.254856 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f82a8-e80c-4949-a2c4-e08f516c06a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:59.263487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f82a8-e80c-4949-a2c4-e08f516c06a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:59.263979 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516f82a8-e80c-4949-a2c4-e08f516c06a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:59.264976 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f82a8-e80c-4949-a2c4-e08f516c06a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:59.267404 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516f82a8-e80c-4949-a2c4-e08f516c06a1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:59.407670 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1086/2179] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:04:58 2022] POST /identity/v3/projects => generated 355 bytes in 487 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:59.434399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97717437-a29a-4231-9abc-25af0e64e400 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:59.535337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97717437-a29a-4231-9abc-25af0e64e400 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:59.542787 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbcc1f-0536-425d-8052-85de076979ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:59.544016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbcc1f-0536-425d-8052-85de076979ad admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:59.544412 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbcc1f-0536-425d-8052-85de076979ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:59.544928 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebbcc1f-0536-425d-8052-85de076979ad admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:59.545959 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbcc1f-0536-425d-8052-85de076979ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:59.549808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebbcc1f-0536-425d-8052-85de076979ad admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:59.700691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1094/2180] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:59 2022] POST /identity/v3/users => generated 286 bytes in 590 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:59.702473 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f82a8-e80c-4949-a2c4-e08f516c06a1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:59.706799 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-516f82a8-e80c-4949-a2c4-e08f516c06a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:04:59.729797 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516f82a8-e80c-4949-a2c4-e08f516c06a1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:59.738267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:59.744666 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:59.744666 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:59.744666 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058 admin admin] PATH_INFO: `/v3/projects/7798d7c788f04d099151350cf2fb1097` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:59.744666 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7798d7c788f04d099151350cf2fb1097)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:59.745306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:59.755759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1087/2181] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:04:59 2022] POST /identity/v3/users => generated 284 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:04:59.757289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbcc1f-0536-425d-8052-85de076979ad admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:59.770415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebbcc1f-0536-425d-8052-85de076979ad admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:59.776965 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7b93f-9e75-4d4e-b2ba-27d6ae99a14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:59.777918 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7b93f-9e75-4d4e-b2ba-27d6ae99a14f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:59.778381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7b93f-9e75-4d4e-b2ba-27d6ae99a14f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:59.778826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa7b93f-9e75-4d4e-b2ba-27d6ae99a14f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:59.779966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7b93f-9e75-4d4e-b2ba-27d6ae99a14f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:59.782849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa7b93f-9e75-4d4e-b2ba-27d6ae99a14f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:59.804051 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1088/2182] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:59.805189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7b93f-9e75-4d4e-b2ba-27d6ae99a14f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:59.821633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa7b93f-9e75-4d4e-b2ba-27d6ae99a14f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:59.829484 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfb27e-3bed-4bb0-af32-66e7ebcb3ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:59.830442 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfb27e-3bed-4bb0-af32-66e7ebcb3ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:59.832168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfb27e-3bed-4bb0-af32-66e7ebcb3ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:59.832168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdfb27e-3bed-4bb0-af32-66e7ebcb3ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:59.832471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfb27e-3bed-4bb0-af32-66e7ebcb3ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:59.835330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdfb27e-3bed-4bb0-af32-66e7ebcb3ca8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:59.854680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1089/2183] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:59.855470 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfb27e-3bed-4bb0-af32-66e7ebcb3ca8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:59.868540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdfb27e-3bed-4bb0-af32-66e7ebcb3ca8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:59.874098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9462f2-13f1-47d1-95b8-bf35166e5ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:59.878580 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9462f2-13f1-47d1-95b8-bf35166e5ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:59.878580 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9462f2-13f1-47d1-95b8-bf35166e5ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:59.878580 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9462f2-13f1-47d1-95b8-bf35166e5ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:59.878580 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9462f2-13f1-47d1-95b8-bf35166e5ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:59.878580 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9462f2-13f1-47d1-95b8-bf35166e5ca1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:59.890459 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1090/2184] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:04:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:04:59.891293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9462f2-13f1-47d1-95b8-bf35166e5ca1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:04:59.901829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9462f2-13f1-47d1-95b8-bf35166e5ca1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:04:59.906778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a96bf-863c-45ed-9b17-f274bc3b8aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:04:59.907371 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a96bf-863c-45ed-9b17-f274bc3b8aa2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:04:59.907707 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a96bf-863c-45ed-9b17-f274bc3b8aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:04:59.907939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a96bf-863c-45ed-9b17-f274bc3b8aa2 admin admin] PATH_INFO: `/v3/projects/ac4bc07f889d4394a9ac2ffdea6d8582/users/8cec29c4515345a88fea1795fdaa4d49/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:04:59.924232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a96bf-863c-45ed-9b17-f274bc3b8aa2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac4bc07f889d4394a9ac2ffdea6d8582, user_id=8cec29c4515345a88fea1795fdaa4d49, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:04:59.927583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a96bf-863c-45ed-9b17-f274bc3b8aa2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:04:59.952342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7798d7c788f04d099151350cf2fb1097', 'request_id': 'req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:04:59.961342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7798d7c788f04d099151350cf2fb1097', 'request_id': 'req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:00.079263 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1091/2185] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:04:59 2022] PUT /identity/v3/projects/ac4bc07f889d4394a9ac2ffdea6d8582/users/8cec29c4515345a88fea1795fdaa4d49/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:00.080127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a96bf-863c-45ed-9b17-f274bc3b8aa2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:00.131437 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7798d7c788f04d099151350cf2fb1097', 'request_id': 'req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:00.153420 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058 admin admin] The token cache is being invalidate because project 7798d7c788f04d099151350cf2fb1097 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:00.154003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:00.158646 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:00.162824 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1095/2186] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:04:59 2022] DELETE /identity/v3/projects/7798d7c788f04d099151350cf2fb1097 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:00.165805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:00.168388 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:00.180426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a96bf-863c-45ed-9b17-f274bc3b8aa2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:00.191858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea6deb-294f-4398-8a1d-2b9fb8f08c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:00.192704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea6deb-294f-4398-8a1d-2b9fb8f08c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:00.193853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea6deb-294f-4398-8a1d-2b9fb8f08c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:00.194477 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea6deb-294f-4398-8a1d-2b9fb8f08c43 admin admin] PATH_INFO: `/v3/projects/31e42dd8f6ff445cb7d44b0b0ce94fc2/users/349bc0a30f324816804e596c937e3da0/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:00.233518 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea6deb-294f-4398-8a1d-2b9fb8f08c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31e42dd8f6ff445cb7d44b0b0ce94fc2, user_id=349bc0a30f324816804e596c937e3da0, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:00.242993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea6deb-294f-4398-8a1d-2b9fb8f08c43 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:00.297609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f8b56-f4f2-4f3a-b29d-d857e07f1058 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:00.304204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd1972-8118-473c-beb1-153f0d436149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:00.305070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd1972-8118-473c-beb1-153f0d436149 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:00.305219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd1972-8118-473c-beb1-153f0d436149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:00.305502 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd1972-8118-473c-beb1-153f0d436149 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:00.306173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd1972-8118-473c-beb1-153f0d436149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:00.307849 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd1972-8118-473c-beb1-153f0d436149 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:00.322276 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1096/2187] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:00 2022] GET /identity/v3/roles => generated 1420 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:00.323708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387156ee-cf18-4fa0-9c1d-6b62f38772d2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:00.324002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387156ee-cf18-4fa0-9c1d-6b62f38772d2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:00.324278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387156ee-cf18-4fa0-9c1d-6b62f38772d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:00.347804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-387156ee-cf18-4fa0-9c1d-6b62f38772d2 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:00.349506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-387156ee-cf18-4fa0-9c1d-6b62f38772d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:00.359343 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1097/2188] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:00.364170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387156ee-cf18-4fa0-9c1d-6b62f38772d2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:00.377415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387156ee-cf18-4fa0-9c1d-6b62f38772d2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:00.383240 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d78b4-f3c1-42ad-a29e-036d370cecb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:00.384132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d78b4-f3c1-42ad-a29e-036d370cecb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:00.384822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d78b4-f3c1-42ad-a29e-036d370cecb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:00.385295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d78b4-f3c1-42ad-a29e-036d370cecb3 admin admin] PATH_INFO: `/v3/projects/ac4bc07f889d4394a9ac2ffdea6d8582/users/8cec29c4515345a88fea1795fdaa4d49/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:00.387584 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d78b4-f3c1-42ad-a29e-036d370cecb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac4bc07f889d4394a9ac2ffdea6d8582, user_id=8cec29c4515345a88fea1795fdaa4d49, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:00.390482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6d78b4-f3c1-42ad-a29e-036d370cecb3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:00.402614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1098/2189] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:00 2022] PUT /identity/v3/projects/ac4bc07f889d4394a9ac2ffdea6d8582/users/8cec29c4515345a88fea1795fdaa4d49/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:00.404050 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044261a-ab8a-4fc8-89ef-09203dce153b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:00.404376 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044261a-ab8a-4fc8-89ef-09203dce153b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:00.404811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5044261a-ab8a-4fc8-89ef-09203dce153b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:00.429838 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5044261a-ab8a-4fc8-89ef-09203dce153b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:00.431450 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5044261a-ab8a-4fc8-89ef-09203dce153b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:00.453294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1092/2190] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:00 2022] PUT /identity/v3/projects/31e42dd8f6ff445cb7d44b0b0ce94fc2/users/349bc0a30f324816804e596c937e3da0/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:00.454605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33170677-ece4-499a-b02e-f4302144077d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:00.454911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33170677-ece4-499a-b02e-f4302144077d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:00.455153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33170677-ece4-499a-b02e-f4302144077d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:00.510372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-33170677-ece4-499a-b02e-f4302144077d None None] MFA Rules not processed for user `8cec29c4515345a88fea1795fdaa4d49`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:00.510372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-33170677-ece4-499a-b02e-f4302144077d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:00.532821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1099/2191] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:00.536063 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e3ccd-6b7b-4425-81bd-55aac98185fe None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:00.541174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e3ccd-6b7b-4425-81bd-55aac98185fe None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:00.541174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10e3ccd-6b7b-4425-81bd-55aac98185fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:00.570994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b10e3ccd-6b7b-4425-81bd-55aac98185fe None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:00.572878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b10e3ccd-6b7b-4425-81bd-55aac98185fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:00.631087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1093/2192] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:00 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:00.631087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33170677-ece4-499a-b02e-f4302144077d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:00.679340 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1100/2193] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:00.684243 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e3ccd-6b7b-4425-81bd-55aac98185fe None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:00.684243 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b10e3ccd-6b7b-4425-81bd-55aac98185fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:00.725953 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33170677-ece4-499a-b02e-f4302144077d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:00.732173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d5455-b02a-46cf-8206-d2702113aab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:00.732800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d5455-b02a-46cf-8206-d2702113aab0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:00.733082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d5455-b02a-46cf-8206-d2702113aab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:00.733357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d5455-b02a-46cf-8206-d2702113aab0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:00.734057 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d5455-b02a-46cf-8206-d2702113aab0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:05:00.734361 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d5455-b02a-46cf-8206-d2702113aab0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:00.736949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5d5455-b02a-46cf-8206-d2702113aab0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:00.755526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1094/2194] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:05:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:05:00.756505 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d5455-b02a-46cf-8206-d2702113aab0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:00.758435 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6b5d5455-b02a-46cf-8206-d2702113aab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:00.787845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10e3ccd-6b7b-4425-81bd-55aac98185fe None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:00.792508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd769ce-4289-472d-ad2f-f38b258a7b26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:00.793141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd769ce-4289-472d-ad2f-f38b258a7b26 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:00.793448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd769ce-4289-472d-ad2f-f38b258a7b26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:00.793700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd769ce-4289-472d-ad2f-f38b258a7b26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:00.805347 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd769ce-4289-472d-ad2f-f38b258a7b26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:00.806844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd769ce-4289-472d-ad2f-f38b258a7b26 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:00.823843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1101/2195] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:00 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:00.825199 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd769ce-4289-472d-ad2f-f38b258a7b26 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:00.842296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd769ce-4289-472d-ad2f-f38b258a7b26 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:00.848182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7248c5-3a9d-4494-8702-61bb7904ba89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:00.848794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7248c5-3a9d-4494-8702-61bb7904ba89 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:00.849100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d5455-b02a-46cf-8206-d2702113aab0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:00.849269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7248c5-3a9d-4494-8702-61bb7904ba89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:00.849469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7248c5-3a9d-4494-8702-61bb7904ba89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:00.850044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7248c5-3a9d-4494-8702-61bb7904ba89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:00.852037 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7248c5-3a9d-4494-8702-61bb7904ba89 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:00.853989 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33639c3-45cc-4f92-8e1d-fb22ede19cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:00.854930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33639c3-45cc-4f92-8e1d-fb22ede19cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:00.855429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33639c3-45cc-4f92-8e1d-fb22ede19cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:00.855975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33639c3-45cc-4f92-8e1d-fb22ede19cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:00.871191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33639c3-45cc-4f92-8e1d-fb22ede19cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:00.872828 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33639c3-45cc-4f92-8e1d-fb22ede19cf2 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:00.886531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1095/2196] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:01.062779 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1102/2197] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:05:00 2022] POST /identity/v3/projects => generated 365 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:01.072790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33639c3-45cc-4f92-8e1d-fb22ede19cf2 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:01.087595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33639c3-45cc-4f92-8e1d-fb22ede19cf2 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:01.149905 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7248c5-3a9d-4494-8702-61bb7904ba89 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:01.151919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fb7248c5-3a9d-4494-8702-61bb7904ba89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:01.190179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c006fd-fa8f-4e70-99e1-403bf650d5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:01.192049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c006fd-fa8f-4e70-99e1-403bf650d5ec admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:01.192613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c006fd-fa8f-4e70-99e1-403bf650d5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:01.192891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c006fd-fa8f-4e70-99e1-403bf650d5ec admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:01.193749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c006fd-fa8f-4e70-99e1-403bf650d5ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:01.196477 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c006fd-fa8f-4e70-99e1-403bf650d5ec admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:01.265327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7248c5-3a9d-4494-8702-61bb7904ba89 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:01.275380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663536b6-fd56-4566-8cd3-6c7896108c73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:01.276049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663536b6-fd56-4566-8cd3-6c7896108c73 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:01.276332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663536b6-fd56-4566-8cd3-6c7896108c73 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:01.276632 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663536b6-fd56-4566-8cd3-6c7896108c73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:01.278541 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-663536b6-fd56-4566-8cd3-6c7896108c73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:01.338280 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1096/2198] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:05:01 2022] POST /identity/v3/users => generated 283 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:01.342744 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c006fd-fa8f-4e70-99e1-403bf650d5ec admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:01.349223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663536b6-fd56-4566-8cd3-6c7896108c73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:01.350711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663536b6-fd56-4566-8cd3-6c7896108c73 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:01.357124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c006fd-fa8f-4e70-99e1-403bf650d5ec admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:01.360646 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1103/2199] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:05:01.363040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90539390-24ce-450f-86f4-1eeb0d8263ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:01.364078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90539390-24ce-450f-86f4-1eeb0d8263ea admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:01.364559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90539390-24ce-450f-86f4-1eeb0d8263ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:01.364917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90539390-24ce-450f-86f4-1eeb0d8263ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:01.365655 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90539390-24ce-450f-86f4-1eeb0d8263ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:01.367420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90539390-24ce-450f-86f4-1eeb0d8263ea admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:01.390252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1097/2200] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:05:01.393161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663536b6-fd56-4566-8cd3-6c7896108c73 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:01.417261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663536b6-fd56-4566-8cd3-6c7896108c73 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:01.422789 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87c84a-626a-4e3f-83a4-ba61b1fea29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:01.423419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87c84a-626a-4e3f-83a4-ba61b1fea29c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:01.423734 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87c84a-626a-4e3f-83a4-ba61b1fea29c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:01.424013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b87c84a-626a-4e3f-83a4-ba61b1fea29c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:01.424935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87c84a-626a-4e3f-83a4-ba61b1fea29c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:01.427141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b87c84a-626a-4e3f-83a4-ba61b1fea29c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:01.440571 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1104/2201] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:01 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:05:01.445338 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90539390-24ce-450f-86f4-1eeb0d8263ea admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:01.457192 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90539390-24ce-450f-86f4-1eeb0d8263ea admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:01.470754 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553250d-772b-4845-9fb0-dfaa762bdb3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:01.495240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553250d-772b-4845-9fb0-dfaa762bdb3d admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:01.495240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553250d-772b-4845-9fb0-dfaa762bdb3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:01.495240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553250d-772b-4845-9fb0-dfaa762bdb3d admin admin] PATH_INFO: `/v3/projects/762f4bf5996e4da99f2a9bce719999c9/users/c742b7e0fac941a6ba58266bfc3e1440/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:01.495527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553250d-772b-4845-9fb0-dfaa762bdb3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=762f4bf5996e4da99f2a9bce719999c9, user_id=c742b7e0fac941a6ba58266bfc3e1440, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:01.502808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553250d-772b-4845-9fb0-dfaa762bdb3d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:01.517455 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87c84a-626a-4e3f-83a4-ba61b1fea29c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:01.519373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8b87c84a-626a-4e3f-83a4-ba61b1fea29c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:01.535992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b87c84a-626a-4e3f-83a4-ba61b1fea29c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:01.543775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64109b19-fd30-45be-8a15-fe2e5545ed2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:01.544540 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64109b19-fd30-45be-8a15-fe2e5545ed2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:01.544908 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64109b19-fd30-45be-8a15-fe2e5545ed2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:01.545291 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64109b19-fd30-45be-8a15-fe2e5545ed2e admin admin] PATH_INFO: `/v3/users/3b460ad3d2214c6d8f25c16172a26bb7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:01.546658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64109b19-fd30-45be-8a15-fe2e5545ed2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b460ad3d2214c6d8f25c16172a26bb7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:01.549438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64109b19-fd30-45be-8a15-fe2e5545ed2e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:01.718026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1098/2202] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:01 2022] PUT /identity/v3/projects/762f4bf5996e4da99f2a9bce719999c9/users/c742b7e0fac941a6ba58266bfc3e1440/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:05:01.718657 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553250d-772b-4845-9fb0-dfaa762bdb3d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:01.720384 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b553250d-772b-4845-9fb0-dfaa762bdb3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:01.838999 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553250d-772b-4845-9fb0-dfaa762bdb3d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:01.846964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2198b7a-b514-491f-b04c-22922c40d89c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:01.848099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2198b7a-b514-491f-b04c-22922c40d89c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:01.848557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2198b7a-b514-491f-b04c-22922c40d89c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:01.848933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2198b7a-b514-491f-b04c-22922c40d89c admin admin] PATH_INFO: `/v3/users/32cc9d07c1e84d47b07832abd3b4a6e8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:01.850662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2198b7a-b514-491f-b04c-22922c40d89c admin admin] RBAC: Authorizing `identity:delete_user(user_id=32cc9d07c1e84d47b07832abd3b4a6e8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:01.853767 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2198b7a-b514-491f-b04c-22922c40d89c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:02.041278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-64109b19-fd30-45be-8a15-fe2e5545ed2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b460ad3d2214c6d8f25c16172a26bb7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:02.194694 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=2,keystone:SELECT=93,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:02.247983 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-64109b19-fd30-45be-8a15-fe2e5545ed2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b460ad3d2214c6d8f25c16172a26bb7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:02.248612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-64109b19-fd30-45be-8a15-fe2e5545ed2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b460ad3d2214c6d8f25c16172a26bb7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:02.249297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-64109b19-fd30-45be-8a15-fe2e5545ed2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b460ad3d2214c6d8f25c16172a26bb7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:02.274340 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-64109b19-fd30-45be-8a15-fe2e5545ed2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b460ad3d2214c6d8f25c16172a26bb7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:02.295520 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1105/2203] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:05:01 2022] DELETE /identity/v3/users/3b460ad3d2214c6d8f25c16172a26bb7 => generated 0 bytes in 779 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:05:02.303488 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64da52-1073-4d8a-951f-c3320674e5eb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:02.304018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64da52-1073-4d8a-951f-c3320674e5eb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:02.304723 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab64da52-1073-4d8a-951f-c3320674e5eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:02.325249 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:02.344615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ab64da52-1073-4d8a-951f-c3320674e5eb None None] MFA Rules not processed for user `c742b7e0fac941a6ba58266bfc3e1440`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:02.345997 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ab64da52-1073-4d8a-951f-c3320674e5eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:02.428747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1106/2204] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:02.430534 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb486510-dc3a-4fe9-9f5f-ae3f1ea4993b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:02.430946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb486510-dc3a-4fe9-9f5f-ae3f1ea4993b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:02.431212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb486510-dc3a-4fe9-9f5f-ae3f1ea4993b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:02.460131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fb486510-dc3a-4fe9-9f5f-ae3f1ea4993b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:02.462190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fb486510-dc3a-4fe9-9f5f-ae3f1ea4993b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:02.463475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a2198b7a-b514-491f-b04c-22922c40d89c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32cc9d07c1e84d47b07832abd3b4a6e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:02.547745 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1107/2205] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:02.550025 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb486510-dc3a-4fe9-9f5f-ae3f1ea4993b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:02.552723 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fb486510-dc3a-4fe9-9f5f-ae3f1ea4993b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:02.632224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a2198b7a-b514-491f-b04c-22922c40d89c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32cc9d07c1e84d47b07832abd3b4a6e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:02.633186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a2198b7a-b514-491f-b04c-22922c40d89c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32cc9d07c1e84d47b07832abd3b4a6e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:02.634100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a2198b7a-b514-491f-b04c-22922c40d89c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32cc9d07c1e84d47b07832abd3b4a6e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:02.636595 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb486510-dc3a-4fe9-9f5f-ae3f1ea4993b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:02.642768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ede2277-b17a-405a-9090-244f8c6dee08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:02.643315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede2277-b17a-405a-9090-244f8c6dee08 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:02.646241 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede2277-b17a-405a-9090-244f8c6dee08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:02.646241 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ede2277-b17a-405a-9090-244f8c6dee08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:02.648457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a2198b7a-b514-491f-b04c-22922c40d89c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32cc9d07c1e84d47b07832abd3b4a6e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:02.656671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7ede2277-b17a-405a-9090-244f8c6dee08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:02.670774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede2277-b17a-405a-9090-244f8c6dee08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:02.672320 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ede2277-b17a-405a-9090-244f8c6dee08 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:02.677593 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1099/2206] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:05:01 2022] DELETE /identity/v3/users/32cc9d07c1e84d47b07832abd3b4a6e8 => generated 0 bytes in 954 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:02.677593 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2198b7a-b514-491f-b04c-22922c40d89c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:02.677593 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a2198b7a-b514-491f-b04c-22922c40d89c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:02.683677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1108/2207] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:02.685088 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14e0b6-6aac-4f0f-be24-56c57c36a870 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:02.685441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14e0b6-6aac-4f0f-be24-56c57c36a870 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:02.685725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14e0b6-6aac-4f0f-be24-56c57c36a870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:02.722538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bf14e0b6-6aac-4f0f-be24-56c57c36a870 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:02.730370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bf14e0b6-6aac-4f0f-be24-56c57c36a870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:02.842494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2198b7a-b514-491f-b04c-22922c40d89c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:02.850468 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050a0d2-bb33-478f-af45-bd6a36896b90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:02.851138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050a0d2-bb33-478f-af45-bd6a36896b90 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:02.851422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050a0d2-bb33-478f-af45-bd6a36896b90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:02.851723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8050a0d2-bb33-478f-af45-bd6a36896b90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:02.854961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8050a0d2-bb33-478f-af45-bd6a36896b90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:02.887131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1109/2208] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:02.900494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14e0b6-6aac-4f0f-be24-56c57c36a870 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:02.918199 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14e0b6-6aac-4f0f-be24-56c57c36a870 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:02.931084 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f442e58-2ad9-4218-b5e1-fc3d76405500 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:02.931756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f442e58-2ad9-4218-b5e1-fc3d76405500 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:02.932026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f442e58-2ad9-4218-b5e1-fc3d76405500 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:02.932644 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f442e58-2ad9-4218-b5e1-fc3d76405500 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:02.950430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f442e58-2ad9-4218-b5e1-fc3d76405500 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:02.952756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f442e58-2ad9-4218-b5e1-fc3d76405500 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:02.956233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050a0d2-bb33-478f-af45-bd6a36896b90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:02.957753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8050a0d2-bb33-478f-af45-bd6a36896b90 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:02.966478 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1110/2209] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:05:02.969963 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1100/2210] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:03.512946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31276c1c-495a-4835-b7b2-6e8fd0371a9e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:03.513665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31276c1c-495a-4835-b7b2-6e8fd0371a9e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:03.513665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31276c1c-495a-4835-b7b2-6e8fd0371a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:03.615306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-31276c1c-495a-4835-b7b2-6e8fd0371a9e None None] MFA Rules not processed for user `349bc0a30f324816804e596c937e3da0`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:03.618762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-31276c1c-495a-4835-b7b2-6e8fd0371a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:03.803895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1111/2211] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:05:03.930193 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050a0d2-bb33-478f-af45-bd6a36896b90 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:03.942348 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31276c1c-495a-4835-b7b2-6e8fd0371a9e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:03.942465 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8050a0d2-bb33-478f-af45-bd6a36896b90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:03.944580 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-31276c1c-495a-4835-b7b2-6e8fd0371a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:03.959010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8050a0d2-bb33-478f-af45-bd6a36896b90 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:03.966657 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c985bd9-0c96-47e4-b9db-9ca55bded6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:03.969388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c985bd9-0c96-47e4-b9db-9ca55bded6db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:03.970109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c985bd9-0c96-47e4-b9db-9ca55bded6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:03.970714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c985bd9-0c96-47e4-b9db-9ca55bded6db admin admin] PATH_INFO: `/v3/projects/b3205249ee3b498cb112f347a4c1f7ce` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:03.975829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31276c1c-495a-4835-b7b2-6e8fd0371a9e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:04.000082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c985bd9-0c96-47e4-b9db-9ca55bded6db admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3205249ee3b498cb112f347a4c1f7ce)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:04.000228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:04.000228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:04.000228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:04.000228 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9 admin admin] PATH_INFO: `/v3/projects/a0ed0a3eace34b6ab24467e31362308f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:04.000823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0ed0a3eace34b6ab24467e31362308f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:04.000976 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c985bd9-0c96-47e4-b9db-9ca55bded6db admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:04.001124 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:04.157631 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1c985bd9-0c96-47e4-b9db-9ca55bded6db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3205249ee3b498cb112f347a4c1f7ce', 'request_id': 'req-1c985bd9-0c96-47e4-b9db-9ca55bded6db'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:04.172652 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1c985bd9-0c96-47e4-b9db-9ca55bded6db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3205249ee3b498cb112f347a4c1f7ce', 'request_id': 'req-1c985bd9-0c96-47e4-b9db-9ca55bded6db'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:04.194463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0ed0a3eace34b6ab24467e31362308f', 'request_id': 'req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:04.202724 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0ed0a3eace34b6ab24467e31362308f', 'request_id': 'req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:04.255203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1c985bd9-0c96-47e4-b9db-9ca55bded6db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3205249ee3b498cb112f347a4c1f7ce', 'request_id': 'req-1c985bd9-0c96-47e4-b9db-9ca55bded6db'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:04.274782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1c985bd9-0c96-47e4-b9db-9ca55bded6db admin admin] The token cache is being invalidate because project b3205249ee3b498cb112f347a4c1f7ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:04.275235 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1c985bd9-0c96-47e4-b9db-9ca55bded6db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:04.275921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1c985bd9-0c96-47e4-b9db-9ca55bded6db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:04.277609 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1101/2212] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:05:03 2022] DELETE /identity/v3/projects/b3205249ee3b498cb112f347a4c1f7ce => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:04.278439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c985bd9-0c96-47e4-b9db-9ca55bded6db admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:04.279862 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1c985bd9-0c96-47e4-b9db-9ca55bded6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:04.352171 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0ed0a3eace34b6ab24467e31362308f', 'request_id': 'req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:04.371266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c985bd9-0c96-47e4-b9db-9ca55bded6db admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:04.378382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9 admin admin] The token cache is being invalidate because project a0ed0a3eace34b6ab24467e31362308f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:04.378382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:04.385961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:04.385961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1112/2213] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:05:03 2022] DELETE /identity/v3/projects/a0ed0a3eace34b6ab24467e31362308f => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:04.385961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:04.385961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:04.487847 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34d7f6-60f4-4f5d-b9fb-e1f81294d6d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:04.489288 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34d7f6-60f4-4f5d-b9fb-e1f81294d6d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:04.490659 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34d7f6-60f4-4f5d-b9fb-e1f81294d6d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:04.491200 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d34d7f6-60f4-4f5d-b9fb-e1f81294d6d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:04.495341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7d34d7f6-60f4-4f5d-b9fb-e1f81294d6d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:04.550830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ab8c03-c0a0-4fde-b98c-87eef733cfa9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:04.561868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acc10b-810e-4ed9-885e-daca9c1a2a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:04.567110 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acc10b-810e-4ed9-885e-daca9c1a2a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:04.571296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acc10b-810e-4ed9-885e-daca9c1a2a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:04.571296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9acc10b-810e-4ed9-885e-daca9c1a2a89 admin admin] PATH_INFO: `/v3/users/10a1d4c10ef04b5a81cbc687ec61be01` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:04.571296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acc10b-810e-4ed9-885e-daca9c1a2a89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10a1d4c10ef04b5a81cbc687ec61be01)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:04.573639 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9acc10b-810e-4ed9-885e-daca9c1a2a89 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:04.597248 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34d7f6-60f4-4f5d-b9fb-e1f81294d6d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:04.599894 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d34d7f6-60f4-4f5d-b9fb-e1f81294d6d3 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:04.612726 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1102/2214] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:04.613897 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34d7f6-60f4-4f5d-b9fb-e1f81294d6d3 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:04.615728 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7d34d7f6-60f4-4f5d-b9fb-e1f81294d6d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:04.639470 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d34d7f6-60f4-4f5d-b9fb-e1f81294d6d3 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:04.649296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ca28e-96a7-4b35-8ed2-40021220da10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:04.650369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ca28e-96a7-4b35-8ed2-40021220da10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:04.650877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ca28e-96a7-4b35-8ed2-40021220da10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:04.651362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ca28e-96a7-4b35-8ed2-40021220da10 admin admin] PATH_INFO: `/v3/users/bffac5553f734e82918d9e8f9087ea7b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:04.653720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ca28e-96a7-4b35-8ed2-40021220da10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bffac5553f734e82918d9e8f9087ea7b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:04.657342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ca28e-96a7-4b35-8ed2-40021220da10 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:05.000410 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-334ca28e-96a7-4b35-8ed2-40021220da10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bffac5553f734e82918d9e8f9087ea7b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:05.002474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c9acc10b-810e-4ed9-885e-daca9c1a2a89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10a1d4c10ef04b5a81cbc687ec61be01'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:05.204249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c9acc10b-810e-4ed9-885e-daca9c1a2a89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10a1d4c10ef04b5a81cbc687ec61be01'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:05.204963 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c9acc10b-810e-4ed9-885e-daca9c1a2a89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10a1d4c10ef04b5a81cbc687ec61be01'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:05.205600 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c9acc10b-810e-4ed9-885e-daca9c1a2a89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10a1d4c10ef04b5a81cbc687ec61be01'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:05.221012 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c9acc10b-810e-4ed9-885e-daca9c1a2a89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10a1d4c10ef04b5a81cbc687ec61be01'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:05.235724 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1113/2215] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:05:04 2022] DELETE /identity/v3/users/10a1d4c10ef04b5a81cbc687ec61be01 => generated 0 bytes in 853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:05.335578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-334ca28e-96a7-4b35-8ed2-40021220da10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bffac5553f734e82918d9e8f9087ea7b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:05.346274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-334ca28e-96a7-4b35-8ed2-40021220da10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bffac5553f734e82918d9e8f9087ea7b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:05.346274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-334ca28e-96a7-4b35-8ed2-40021220da10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bffac5553f734e82918d9e8f9087ea7b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:05.359225 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-334ca28e-96a7-4b35-8ed2-40021220da10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bffac5553f734e82918d9e8f9087ea7b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:05.395002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1103/2216] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:05:04 2022] DELETE /identity/v3/users/bffac5553f734e82918d9e8f9087ea7b => generated 0 bytes in 782 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:06.186261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acc10b-810e-4ed9-885e-daca9c1a2a89 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:06.189690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c9acc10b-810e-4ed9-885e-daca9c1a2a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:06.326757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9acc10b-810e-4ed9-885e-daca9c1a2a89 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:06.405619 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ca28e-96a7-4b35-8ed2-40021220da10 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:06.407799 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92a133-c80c-4df4-b44e-b5813affa3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:06.408100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-334ca28e-96a7-4b35-8ed2-40021220da10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:06.408656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92a133-c80c-4df4-b44e-b5813affa3e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:06.409551 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92a133-c80c-4df4-b44e-b5813affa3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:06.410129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92a133-c80c-4df4-b44e-b5813affa3e0 admin admin] PATH_INFO: `/v3/projects/1d720749d3b740a9a28ddd3c71b3ff49` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:06.412434 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92a133-c80c-4df4-b44e-b5813affa3e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d720749d3b740a9a28ddd3c71b3ff49)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:06.414671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92a133-c80c-4df4-b44e-b5813affa3e0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:06.425409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ca28e-96a7-4b35-8ed2-40021220da10 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:06.439753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c20a7-79fb-4efa-a71e-33a68a4811a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:06.440756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c20a7-79fb-4efa-a71e-33a68a4811a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:06.441229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c20a7-79fb-4efa-a71e-33a68a4811a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:06.441661 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c20a7-79fb-4efa-a71e-33a68a4811a4 admin admin] PATH_INFO: `/v3/projects/4ac92811ebcb4b68a3d889e7aad244f9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:06.443255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c20a7-79fb-4efa-a71e-33a68a4811a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ac92811ebcb4b68a3d889e7aad244f9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:06.445541 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c20a7-79fb-4efa-a71e-33a68a4811a4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:06.570721 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:06.693615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-895c20a7-79fb-4efa-a71e-33a68a4811a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ac92811ebcb4b68a3d889e7aad244f9', 'request_id': 'req-895c20a7-79fb-4efa-a71e-33a68a4811a4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:06.697471 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-aa92a133-c80c-4df4-b44e-b5813affa3e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d720749d3b740a9a28ddd3c71b3ff49', 'request_id': 'req-aa92a133-c80c-4df4-b44e-b5813affa3e0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:06.704770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-aa92a133-c80c-4df4-b44e-b5813affa3e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d720749d3b740a9a28ddd3c71b3ff49', 'request_id': 'req-aa92a133-c80c-4df4-b44e-b5813affa3e0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:06.708886 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-895c20a7-79fb-4efa-a71e-33a68a4811a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ac92811ebcb4b68a3d889e7aad244f9', 'request_id': 'req-895c20a7-79fb-4efa-a71e-33a68a4811a4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:06.728533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:06.855203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-aa92a133-c80c-4df4-b44e-b5813affa3e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d720749d3b740a9a28ddd3c71b3ff49', 'request_id': 'req-aa92a133-c80c-4df4-b44e-b5813affa3e0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:06.874641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-aa92a133-c80c-4df4-b44e-b5813affa3e0 admin admin] The token cache is being invalidate because project 1d720749d3b740a9a28ddd3c71b3ff49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:06.875283 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-aa92a133-c80c-4df4-b44e-b5813affa3e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:06.876640 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-aa92a133-c80c-4df4-b44e-b5813affa3e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:06.907123 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1114/2217] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:05:06 2022] DELETE /identity/v3/projects/1d720749d3b740a9a28ddd3c71b3ff49 => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:06.909058 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1d3c9-1af3-4832-ab93-90f5016046bc None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:06.915144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1d3c9-1af3-4832-ab93-90f5016046bc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:06.915144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe1d3c9-1af3-4832-ab93-90f5016046bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:06.950687 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8fe1d3c9-1af3-4832-ab93-90f5016046bc None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:06.952661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8fe1d3c9-1af3-4832-ab93-90f5016046bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:06.994969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-895c20a7-79fb-4efa-a71e-33a68a4811a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ac92811ebcb4b68a3d889e7aad244f9', 'request_id': 'req-895c20a7-79fb-4efa-a71e-33a68a4811a4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:07.014863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-895c20a7-79fb-4efa-a71e-33a68a4811a4 admin admin] The token cache is being invalidate because project 4ac92811ebcb4b68a3d889e7aad244f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:07.015363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-895c20a7-79fb-4efa-a71e-33a68a4811a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:07.017120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-895c20a7-79fb-4efa-a71e-33a68a4811a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:07.021608 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1104/2218] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:05:06 2022] DELETE /identity/v3/projects/4ac92811ebcb4b68a3d889e7aad244f9 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:07.031009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c20a7-79fb-4efa-a71e-33a68a4811a4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:07.034237 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-895c20a7-79fb-4efa-a71e-33a68a4811a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:07.092574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1115/2219] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:07.101357 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e7214-2e9a-4347-ac15-676df4368d95 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:07.101778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e7214-2e9a-4347-ac15-676df4368d95 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:07.101995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4e7214-2e9a-4347-ac15-676df4368d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:07.133717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4b4e7214-2e9a-4347-ac15-676df4368d95 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:07.135227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4b4e7214-2e9a-4347-ac15-676df4368d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:07.146624 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c20a7-79fb-4efa-a71e-33a68a4811a4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:07.148091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1116/2220] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:07.151708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107a945-fca0-4917-8020-01cac2dae0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:07.152273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107a945-fca0-4917-8020-01cac2dae0e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:07.152553 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107a945-fca0-4917-8020-01cac2dae0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:07.152660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e7214-2e9a-4347-ac15-676df4368d95 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:07.152820 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0107a945-fca0-4917-8020-01cac2dae0e3 admin admin] PATH_INFO: `/v3/users/54c4bde5b73e448fab87831d727d4f0c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:07.153955 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107a945-fca0-4917-8020-01cac2dae0e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54c4bde5b73e448fab87831d727d4f0c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:07.156001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0107a945-fca0-4917-8020-01cac2dae0e3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:07.170132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4e7214-2e9a-4347-ac15-676df4368d95 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:07.179486 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc224c-fbd8-4347-864e-28ef37fad27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:07.180515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc224c-fbd8-4347-864e-28ef37fad27e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:07.181128 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc224c-fbd8-4347-864e-28ef37fad27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:07.181439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cc224c-fbd8-4347-864e-28ef37fad27e admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:07.182351 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc224c-fbd8-4347-864e-28ef37fad27e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:05:07.182777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc224c-fbd8-4347-864e-28ef37fad27e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:07.184606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cc224c-fbd8-4347-864e-28ef37fad27e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:07.199244 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1117/2221] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:05:07.205455 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc224c-fbd8-4347-864e-28ef37fad27e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:07.220090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cc224c-fbd8-4347-864e-28ef37fad27e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:07.225498 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9770c-813d-4466-a577-5f5bc065c8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:07.226334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9770c-813d-4466-a577-5f5bc065c8aa admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:07.226514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9770c-813d-4466-a577-5f5bc065c8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:07.226833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f9770c-813d-4466-a577-5f5bc065c8aa admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:07.227507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f9770c-813d-4466-a577-5f5bc065c8aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:07.229463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f9770c-813d-4466-a577-5f5bc065c8aa admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:07.345280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1118/2222] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:05:07 2022] POST /identity/v3/projects => generated 377 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:07.350016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9770c-813d-4466-a577-5f5bc065c8aa admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:07.433491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f9770c-813d-4466-a577-5f5bc065c8aa admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:07.439616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8da450-4463-4883-a1f1-6646cc69d12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:07.441083 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8da450-4463-4883-a1f1-6646cc69d12a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:07.441361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8da450-4463-4883-a1f1-6646cc69d12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:07.441676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8da450-4463-4883-a1f1-6646cc69d12a admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:07.442433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8da450-4463-4883-a1f1-6646cc69d12a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:07.444441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8da450-4463-4883-a1f1-6646cc69d12a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:07.552644 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0107a945-fca0-4917-8020-01cac2dae0e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54c4bde5b73e448fab87831d727d4f0c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:07.635894 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1119/2223] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:05:07 2022] POST /identity/v3/users => generated 289 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:07.635894 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8da450-4463-4883-a1f1-6646cc69d12a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:07.647989 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8da450-4463-4883-a1f1-6646cc69d12a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:07.655820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf3c5f-5be1-475c-ad35-91f56b561df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:07.656451 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf3c5f-5be1-475c-ad35-91f56b561df3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:07.656735 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf3c5f-5be1-475c-ad35-91f56b561df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:07.657008 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf3c5f-5be1-475c-ad35-91f56b561df3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:07.657658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf3c5f-5be1-475c-ad35-91f56b561df3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:07.660538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf3c5f-5be1-475c-ad35-91f56b561df3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:07.666173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0107a945-fca0-4917-8020-01cac2dae0e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54c4bde5b73e448fab87831d727d4f0c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:07.671148 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0107a945-fca0-4917-8020-01cac2dae0e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54c4bde5b73e448fab87831d727d4f0c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:07.671148 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0107a945-fca0-4917-8020-01cac2dae0e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54c4bde5b73e448fab87831d727d4f0c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:07.683096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1120/2224] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:05:07.687180 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf3c5f-5be1-475c-ad35-91f56b561df3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:07.689840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4cdf3c5f-5be1-475c-ad35-91f56b561df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:07.699784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0107a945-fca0-4917-8020-01cac2dae0e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54c4bde5b73e448fab87831d727d4f0c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:07.711825 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf3c5f-5be1-475c-ad35-91f56b561df3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:07.720558 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5182f8ae-061e-45d9-b9f8-d5decad662e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:07.722173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182f8ae-061e-45d9-b9f8-d5decad662e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:07.722644 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182f8ae-061e-45d9-b9f8-d5decad662e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:07.723301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5182f8ae-061e-45d9-b9f8-d5decad662e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:07.725651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182f8ae-061e-45d9-b9f8-d5decad662e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:07.726453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5182f8ae-061e-45d9-b9f8-d5decad662e4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:07.735288 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1105/2225] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:05:07 2022] DELETE /identity/v3/users/54c4bde5b73e448fab87831d727d4f0c => generated 0 bytes in 708 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:07.746038 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1121/2226] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:07.750271 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107a945-fca0-4917-8020-01cac2dae0e3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:07.830081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0107a945-fca0-4917-8020-01cac2dae0e3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:07.837478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fceedf0-5d8b-4acf-b940-554cc9130ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:07.838079 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fceedf0-5d8b-4acf-b940-554cc9130ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:07.838462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fceedf0-5d8b-4acf-b940-554cc9130ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:07.839203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fceedf0-5d8b-4acf-b940-554cc9130ac5 admin admin] PATH_INFO: `/v3/projects/f384fc2a628a48c6b46d021f4dd42beb/users/48526370b77345a4b590ab7b84bf0837/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:07.861648 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fceedf0-5d8b-4acf-b940-554cc9130ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f384fc2a628a48c6b46d021f4dd42beb, user_id=48526370b77345a4b590ab7b84bf0837, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:07.864479 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fceedf0-5d8b-4acf-b940-554cc9130ac5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:07.921703 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:07.995315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1106/2227] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:07 2022] PUT /identity/v3/projects/f384fc2a628a48c6b46d021f4dd42beb/users/48526370b77345a4b590ab7b84bf0837/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:05:08.008369 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205605e-da55-4690-a8cf-98f452bab710 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:08.008656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205605e-da55-4690-a8cf-98f452bab710 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:08.008957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7205605e-da55-4690-a8cf-98f452bab710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:08.019500 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:08.047357 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7205605e-da55-4690-a8cf-98f452bab710 None None] MFA Rules not processed for user `48526370b77345a4b590ab7b84bf0837`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:08.048326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7205605e-da55-4690-a8cf-98f452bab710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:08.162974 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1122/2228] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:08.180014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fceedf0-5d8b-4acf-b940-554cc9130ac5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:08.181837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3fceedf0-5d8b-4acf-b940-554cc9130ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:08.257555 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fceedf0-5d8b-4acf-b940-554cc9130ac5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:08.262030 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f348b71-9824-4d20-b049-85f5321397b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6dee005fc3d4fb593a1771e7851c20a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:08.262778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f348b71-9824-4d20-b049-85f5321397b1 service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:08.263169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f348b71-9824-4d20-b049-85f5321397b1 service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:08.263452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f348b71-9824-4d20-b049-85f5321397b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:08.272943 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f348b71-9824-4d20-b049-85f5321397b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:08.274549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f348b71-9824-4d20-b049-85f5321397b1 service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:08.285225 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1107/2229] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:08.418368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205605e-da55-4690-a8cf-98f452bab710 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:08.419023 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7205605e-da55-4690-a8cf-98f452bab710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:08.529950 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7205605e-da55-4690-a8cf-98f452bab710 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:08.540842 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4eb084-6141-40bb-a01d-f1489dfe4fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:08.540842 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4eb084-6141-40bb-a01d-f1489dfe4fa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:08.540842 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4eb084-6141-40bb-a01d-f1489dfe4fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:08.540842 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4eb084-6141-40bb-a01d-f1489dfe4fa1 admin admin] PATH_INFO: `/v3/projects/e676907015fa483695dbd661846ad887` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:08.542161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4eb084-6141-40bb-a01d-f1489dfe4fa1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e676907015fa483695dbd661846ad887)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:08.545943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4eb084-6141-40bb-a01d-f1489dfe4fa1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:08.656763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82804b-5106-4532-a441-59439690a09c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:08.657694 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82804b-5106-4532-a441-59439690a09c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:08.658030 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a82804b-5106-4532-a441-59439690a09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:08.679223 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8a82804b-5106-4532-a441-59439690a09c None None] MFA Rules not processed for user `349bc0a30f324816804e596c937e3da0`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:08.680421 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8a82804b-5106-4532-a441-59439690a09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:08.783289 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ee4eb084-6141-40bb-a01d-f1489dfe4fa1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e676907015fa483695dbd661846ad887', 'request_id': 'req-ee4eb084-6141-40bb-a01d-f1489dfe4fa1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:08.786406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1108/2230] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:08.796174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ee4eb084-6141-40bb-a01d-f1489dfe4fa1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e676907015fa483695dbd661846ad887', 'request_id': 'req-ee4eb084-6141-40bb-a01d-f1489dfe4fa1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:08.973592 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ee4eb084-6141-40bb-a01d-f1489dfe4fa1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e676907015fa483695dbd661846ad887', 'request_id': 'req-ee4eb084-6141-40bb-a01d-f1489dfe4fa1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:08.991699 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ee4eb084-6141-40bb-a01d-f1489dfe4fa1 admin admin] The token cache is being invalidate because project e676907015fa483695dbd661846ad887 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:08.992160 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ee4eb084-6141-40bb-a01d-f1489dfe4fa1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:08.993594 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ee4eb084-6141-40bb-a01d-f1489dfe4fa1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:08.994847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1123/2231] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:05:08 2022] DELETE /identity/v3/projects/e676907015fa483695dbd661846ad887 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:05:09.000119 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf025d-7012-4bc3-bc5b-0c13df493cc6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:09.000922 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf025d-7012-4bc3-bc5b-0c13df493cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:09.001344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bf025d-7012-4bc3-bc5b-0c13df493cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:09.032900 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f9bf025d-7012-4bc3-bc5b-0c13df493cc6 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:09.035856 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f9bf025d-7012-4bc3-bc5b-0c13df493cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:09.162475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1109/2232] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:05:09.175843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c5332-83f7-47a6-ba86-4d2ae934bfe6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:09.176220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c5332-83f7-47a6-ba86-4d2ae934bfe6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:09.176400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c5332-83f7-47a6-ba86-4d2ae934bfe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:09.222880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-517c5332-83f7-47a6-ba86-4d2ae934bfe6 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:09.225239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-517c5332-83f7-47a6-ba86-4d2ae934bfe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:09.239969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1124/2233] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:09.245446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf025d-7012-4bc3-bc5b-0c13df493cc6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:09.257445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bf025d-7012-4bc3-bc5b-0c13df493cc6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:09.263614 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b330b9a-947f-4cef-a9a9-4d75fd12184e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:09.264403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b330b9a-947f-4cef-a9a9-4d75fd12184e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:09.264955 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b330b9a-947f-4cef-a9a9-4d75fd12184e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:09.265341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b330b9a-947f-4cef-a9a9-4d75fd12184e admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:09.266942 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b330b9a-947f-4cef-a9a9-4d75fd12184e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:05:09.267748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b330b9a-947f-4cef-a9a9-4d75fd12184e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:09.270078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b330b9a-947f-4cef-a9a9-4d75fd12184e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:09.289799 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c5332-83f7-47a6-ba86-4d2ae934bfe6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:09.292082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1110/2234] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:05:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:05:09.301773 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c5332-83f7-47a6-ba86-4d2ae934bfe6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:09.307713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84030a72-e0ce-48ca-81fd-07ee476bd948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:09.308734 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84030a72-e0ce-48ca-81fd-07ee476bd948 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:09.309012 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84030a72-e0ce-48ca-81fd-07ee476bd948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:09.309528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84030a72-e0ce-48ca-81fd-07ee476bd948 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:09.310992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84030a72-e0ce-48ca-81fd-07ee476bd948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:09.313807 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84030a72-e0ce-48ca-81fd-07ee476bd948 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:09.471879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1125/2235] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:05:09 2022] POST /identity/v3/projects => generated 373 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:09.474636 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b330b9a-947f-4cef-a9a9-4d75fd12184e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:09.486287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b330b9a-947f-4cef-a9a9-4d75fd12184e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:09.585469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a2a38-e60f-4961-aa58-cad7fe097d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:09.586944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a2a38-e60f-4961-aa58-cad7fe097d8d admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:09.587396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a2a38-e60f-4961-aa58-cad7fe097d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:09.587877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8a2a38-e60f-4961-aa58-cad7fe097d8d admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:09.588786 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a2a38-e60f-4961-aa58-cad7fe097d8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:09.595982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8a2a38-e60f-4961-aa58-cad7fe097d8d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:09.877295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1111/2236] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:05:09 2022] POST /identity/v3/users => generated 287 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:09.880489 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84030a72-e0ce-48ca-81fd-07ee476bd948 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:09.914613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84030a72-e0ce-48ca-81fd-07ee476bd948 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:09.921248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f2b37-454e-4d6f-8440-406d273353aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:09.922190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f2b37-454e-4d6f-8440-406d273353aa admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:09.922676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f2b37-454e-4d6f-8440-406d273353aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:09.923150 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f2b37-454e-4d6f-8440-406d273353aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:09.924078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f2b37-454e-4d6f-8440-406d273353aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:09.926597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548f2b37-454e-4d6f-8440-406d273353aa admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:09.944018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1126/2237] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:09.948807 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a2a38-e60f-4961-aa58-cad7fe097d8d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:09.962308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8a2a38-e60f-4961-aa58-cad7fe097d8d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:09.973277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4262b84d-14ec-4875-ac67-360a1d4e1c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:09.974201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262b84d-14ec-4875-ac67-360a1d4e1c90 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:09.974636 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262b84d-14ec-4875-ac67-360a1d4e1c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:09.975092 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4262b84d-14ec-4875-ac67-360a1d4e1c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:09.985217 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262b84d-14ec-4875-ac67-360a1d4e1c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:09.988974 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4262b84d-14ec-4875-ac67-360a1d4e1c90 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:10.017318 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1112/2238] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:10.020596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f2b37-454e-4d6f-8440-406d273353aa admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:10.038952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f2b37-454e-4d6f-8440-406d273353aa admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:10.058719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228dddc9-84c4-452c-997f-fbae1ed7e54c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:10.058719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228dddc9-84c4-452c-997f-fbae1ed7e54c admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:10.058719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228dddc9-84c4-452c-997f-fbae1ed7e54c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:10.058719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228dddc9-84c4-452c-997f-fbae1ed7e54c admin admin] PATH_INFO: `/v3/projects/aa33e03388af4f658531c23d08d2999f/users/f70696d6e8ec4c4a9ceea100e5333b1e/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:10.076211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228dddc9-84c4-452c-997f-fbae1ed7e54c admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa33e03388af4f658531c23d08d2999f, user_id=f70696d6e8ec4c4a9ceea100e5333b1e, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:10.090469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228dddc9-84c4-452c-997f-fbae1ed7e54c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:10.304664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1127/2239] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:10 2022] PUT /identity/v3/projects/aa33e03388af4f658531c23d08d2999f/users/f70696d6e8ec4c4a9ceea100e5333b1e/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:10.323595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ee6d6-8d71-497b-9bac-ca18f486273e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:10.323925 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ee6d6-8d71-497b-9bac-ca18f486273e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:10.324353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882ee6d6-8d71-497b-9bac-ca18f486273e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:10.380069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-882ee6d6-8d71-497b-9bac-ca18f486273e None None] MFA Rules not processed for user `f70696d6e8ec4c4a9ceea100e5333b1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:10.383125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-882ee6d6-8d71-497b-9bac-ca18f486273e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:10.546574 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1113/2240] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:05:10.568388 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228dddc9-84c4-452c-997f-fbae1ed7e54c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:10.571908 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-228dddc9-84c4-452c-997f-fbae1ed7e54c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:10.655282 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228dddc9-84c4-452c-997f-fbae1ed7e54c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:10.661190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0c9550-d910-49cc-955d-60758e4abf19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:10.661849 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c9550-d910-49cc-955d-60758e4abf19 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:10.662153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c9550-d910-49cc-955d-60758e4abf19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:10.662417 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0c9550-d910-49cc-955d-60758e4abf19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:10.676571 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c9550-d910-49cc-955d-60758e4abf19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:10.677975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0c9550-d910-49cc-955d-60758e4abf19 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:10.703076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1128/2241] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:10.740034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ee6d6-8d71-497b-9bac-ca18f486273e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:10.874243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882ee6d6-8d71-497b-9bac-ca18f486273e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:10.886888 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9eabc-706d-447b-9387-5201c21aa690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:10.891175 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9eabc-706d-447b-9387-5201c21aa690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:10.895399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9eabc-706d-447b-9387-5201c21aa690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:10.895740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f9eabc-706d-447b-9387-5201c21aa690 admin admin] PATH_INFO: `/v3/users/f70696d6e8ec4c4a9ceea100e5333b1e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:10.898114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9eabc-706d-447b-9387-5201c21aa690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f70696d6e8ec4c4a9ceea100e5333b1e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:10.901575 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f9eabc-706d-447b-9387-5201c21aa690 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:11.230876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e9f9eabc-706d-447b-9387-5201c21aa690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f70696d6e8ec4c4a9ceea100e5333b1e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:11.405146 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e9f9eabc-706d-447b-9387-5201c21aa690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f70696d6e8ec4c4a9ceea100e5333b1e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:11.406119 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e9f9eabc-706d-447b-9387-5201c21aa690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f70696d6e8ec4c4a9ceea100e5333b1e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:11.406909 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e9f9eabc-706d-447b-9387-5201c21aa690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f70696d6e8ec4c4a9ceea100e5333b1e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:11.419489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e9f9eabc-706d-447b-9387-5201c21aa690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f70696d6e8ec4c4a9ceea100e5333b1e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:11.465846 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1114/2242] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:05:10 2022] DELETE /identity/v3/users/f70696d6e8ec4c4a9ceea100e5333b1e => generated 0 bytes in 727 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:11.471165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea90dbe-2d2a-463c-801d-b49efac143bf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:11.471564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea90dbe-2d2a-463c-801d-b49efac143bf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:11.472270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea90dbe-2d2a-463c-801d-b49efac143bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:11.520335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6ea90dbe-2d2a-463c-801d-b49efac143bf None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:11.523684 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6ea90dbe-2d2a-463c-801d-b49efac143bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:11.711097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1129/2243] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:05:11.724498 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9eabc-706d-447b-9387-5201c21aa690 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:11.728285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e9f9eabc-706d-447b-9387-5201c21aa690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:11.841287 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:11.858554 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f9eabc-706d-447b-9387-5201c21aa690 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:11.866586 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c7bcd4-d220-462f-824f-ccfffbd60dc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:11.867407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7bcd4-d220-462f-824f-ccfffbd60dc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:11.867978 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7bcd4-d220-462f-824f-ccfffbd60dc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:11.867978 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c7bcd4-d220-462f-824f-ccfffbd60dc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:11.890479 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7bcd4-d220-462f-824f-ccfffbd60dc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:11.892489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c7bcd4-d220-462f-824f-ccfffbd60dc0 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:11.910806 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1115/2244] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:11.959334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:13.045030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea90dbe-2d2a-463c-801d-b49efac143bf None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:13.049833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6ea90dbe-2d2a-463c-801d-b49efac143bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:13.073125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea90dbe-2d2a-463c-801d-b49efac143bf None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:13.132069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3609339-ab8b-4c7f-98ab-065f89109132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:13.132069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3609339-ab8b-4c7f-98ab-065f89109132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:13.132327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3609339-ab8b-4c7f-98ab-065f89109132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:13.132841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3609339-ab8b-4c7f-98ab-065f89109132 admin admin] PATH_INFO: `/v3/projects/aa33e03388af4f658531c23d08d2999f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:13.140700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3609339-ab8b-4c7f-98ab-065f89109132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa33e03388af4f658531c23d08d2999f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:13.149409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3609339-ab8b-4c7f-98ab-065f89109132 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:13.423136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a3609339-ab8b-4c7f-98ab-065f89109132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa33e03388af4f658531c23d08d2999f', 'request_id': 'req-a3609339-ab8b-4c7f-98ab-065f89109132'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:13.453917 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a3609339-ab8b-4c7f-98ab-065f89109132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa33e03388af4f658531c23d08d2999f', 'request_id': 'req-a3609339-ab8b-4c7f-98ab-065f89109132'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:13.675516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a3609339-ab8b-4c7f-98ab-065f89109132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa33e03388af4f658531c23d08d2999f', 'request_id': 'req-a3609339-ab8b-4c7f-98ab-065f89109132'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:13.693456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a3609339-ab8b-4c7f-98ab-065f89109132 admin admin] The token cache is being invalidate because project aa33e03388af4f658531c23d08d2999f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:13.693762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a3609339-ab8b-4c7f-98ab-065f89109132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:13.699379 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a3609339-ab8b-4c7f-98ab-065f89109132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:13.716808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1130/2245] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:05:13 2022] DELETE /identity/v3/projects/aa33e03388af4f658531c23d08d2999f => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:13.720150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e05d60-ba33-49aa-8d92-f813efbcb088 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:13.721901 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e05d60-ba33-49aa-8d92-f813efbcb088 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:13.722911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e05d60-ba33-49aa-8d92-f813efbcb088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:13.842848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f8e05d60-ba33-49aa-8d92-f813efbcb088 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:13.848947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:13.851681 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f8e05d60-ba33-49aa-8d92-f813efbcb088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:14.100528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1116/2246] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:14.113284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84335af-c141-48b2-a299-d9565fed8de6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:14.120198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84335af-c141-48b2-a299-d9565fed8de6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:14.120198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84335af-c141-48b2-a299-d9565fed8de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:14.157646 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f84335af-c141-48b2-a299-d9565fed8de6 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:14.163491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f84335af-c141-48b2-a299-d9565fed8de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:14.210878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1131/2247] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:14.215339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e05d60-ba33-49aa-8d92-f813efbcb088 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:14.248551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e05d60-ba33-49aa-8d92-f813efbcb088 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:14.271002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081c434-5b77-4e60-b396-8c1ed619bf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:14.271745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081c434-5b77-4e60-b396-8c1ed619bf57 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:14.272260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081c434-5b77-4e60-b396-8c1ed619bf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:14.272424 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e081c434-5b77-4e60-b396-8c1ed619bf57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:14.273065 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081c434-5b77-4e60-b396-8c1ed619bf57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:05:14.273377 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081c434-5b77-4e60-b396-8c1ed619bf57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:14.275330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e081c434-5b77-4e60-b396-8c1ed619bf57 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:14.335494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84335af-c141-48b2-a299-d9565fed8de6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:14.347434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1117/2248] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:05:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:05:14.365270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84335af-c141-48b2-a299-d9565fed8de6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:14.381360 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff40de-9433-42d0-9987-aa9bebab6006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:14.387301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff40de-9433-42d0-9987-aa9bebab6006 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:14.387301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff40de-9433-42d0-9987-aa9bebab6006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:14.387301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff40de-9433-42d0-9987-aa9bebab6006 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:14.387301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff40de-9433-42d0-9987-aa9bebab6006 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:14.388349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff40de-9433-42d0-9987-aa9bebab6006 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:14.639262 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081c434-5b77-4e60-b396-8c1ed619bf57 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:14.640762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1132/2249] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:05:14 2022] POST /identity/v3/projects => generated 375 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:14.669634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e081c434-5b77-4e60-b396-8c1ed619bf57 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:14.832379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4f0c4-9d1c-4ce8-a19e-3283010ca9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:14.833332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4f0c4-9d1c-4ce8-a19e-3283010ca9c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:14.833989 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4f0c4-9d1c-4ce8-a19e-3283010ca9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:14.834453 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d4f0c4-9d1c-4ce8-a19e-3283010ca9c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:14.835548 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4f0c4-9d1c-4ce8-a19e-3283010ca9c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:14.839627 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d4f0c4-9d1c-4ce8-a19e-3283010ca9c2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:15.013178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1118/2250] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:05:14 2022] POST /identity/v3/users => generated 288 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:15.018665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff40de-9433-42d0-9987-aa9bebab6006 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:15.033905 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff40de-9433-42d0-9987-aa9bebab6006 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:15.043954 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9a557-823f-4ba7-80f5-faa9a5184c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:15.044671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9a557-823f-4ba7-80f5-faa9a5184c5c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:15.044958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9a557-823f-4ba7-80f5-faa9a5184c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:15.054549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9a557-823f-4ba7-80f5-faa9a5184c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:15.054549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9a557-823f-4ba7-80f5-faa9a5184c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:15.054549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9a557-823f-4ba7-80f5-faa9a5184c5c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:15.065287 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1133/2251] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:15.066692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4f0c4-9d1c-4ce8-a19e-3283010ca9c2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:15.078035 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d4f0c4-9d1c-4ce8-a19e-3283010ca9c2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:15.083579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7027b4-7992-445d-804a-1cdab157c4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:15.084256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7027b4-7992-445d-804a-1cdab157c4ef admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:15.084567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7027b4-7992-445d-804a-1cdab157c4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:15.084850 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7027b4-7992-445d-804a-1cdab157c4ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:15.085513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7027b4-7992-445d-804a-1cdab157c4ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:15.087135 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7027b4-7992-445d-804a-1cdab157c4ef admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:15.108773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1119/2252] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:15.120704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9a557-823f-4ba7-80f5-faa9a5184c5c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:15.135343 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9a557-823f-4ba7-80f5-faa9a5184c5c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:15.142158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e81bb1-ff1c-40e7-99a0-b01bc84c7066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:15.142920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e81bb1-ff1c-40e7-99a0-b01bc84c7066 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:15.143210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e81bb1-ff1c-40e7-99a0-b01bc84c7066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:15.143481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e81bb1-ff1c-40e7-99a0-b01bc84c7066 admin admin] PATH_INFO: `/v3/projects/dee85100e2744253bece8a0d74c79f41/users/046e50a17a6a4783895782d3aa2172e2/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:15.163421 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e81bb1-ff1c-40e7-99a0-b01bc84c7066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dee85100e2744253bece8a0d74c79f41, user_id=046e50a17a6a4783895782d3aa2172e2, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:15.166337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e81bb1-ff1c-40e7-99a0-b01bc84c7066 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:15.367560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1134/2253] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:15 2022] PUT /identity/v3/projects/dee85100e2744253bece8a0d74c79f41/users/046e50a17a6a4783895782d3aa2172e2/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:15.394723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7dfd6-55f3-4cae-a981-a5017fa33e0e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:15.394723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7dfd6-55f3-4cae-a981-a5017fa33e0e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:15.394723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d7dfd6-55f3-4cae-a981-a5017fa33e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:15.400323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:15.499097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-49d7dfd6-55f3-4cae-a981-a5017fa33e0e None None] MFA Rules not processed for user `046e50a17a6a4783895782d3aa2172e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:15.501127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-49d7dfd6-55f3-4cae-a981-a5017fa33e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:15.601875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1120/2254] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:15.713382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e81bb1-ff1c-40e7-99a0-b01bc84c7066 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:15.718144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-81e81bb1-ff1c-40e7-99a0-b01bc84c7066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:15.928447 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e81bb1-ff1c-40e7-99a0-b01bc84c7066 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:15.940143 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fcc2a-3d2a-4a07-9652-74a57f7d2e50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:15.940798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fcc2a-3d2a-4a07-9652-74a57f7d2e50 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:15.941080 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fcc2a-3d2a-4a07-9652-74a57f7d2e50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:15.941353 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2fcc2a-3d2a-4a07-9652-74a57f7d2e50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:15.952490 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fcc2a-3d2a-4a07-9652-74a57f7d2e50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:15.954262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2fcc2a-3d2a-4a07-9652-74a57f7d2e50 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:16.009843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1135/2255] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:16.854638 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:17.474750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7dfd6-55f3-4cae-a981-a5017fa33e0e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:17.479553 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-49d7dfd6-55f3-4cae-a981-a5017fa33e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:17.638274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d7dfd6-55f3-4cae-a981-a5017fa33e0e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:17.655311 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e730cece-b817-4691-ab32-96d2e3e8dddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:17.656068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730cece-b817-4691-ab32-96d2e3e8dddc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:17.656368 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730cece-b817-4691-ab32-96d2e3e8dddc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:17.656942 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e730cece-b817-4691-ab32-96d2e3e8dddc admin admin] PATH_INFO: `/v3/users/14e51ac136874806b4b903235697b404` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:17.660622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730cece-b817-4691-ab32-96d2e3e8dddc admin admin] RBAC: Authorizing `identity:delete_user(user_id=14e51ac136874806b4b903235697b404)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:17.664266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e730cece-b817-4691-ab32-96d2e3e8dddc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:18.158304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e730cece-b817-4691-ab32-96d2e3e8dddc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14e51ac136874806b4b903235697b404'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:18.306799 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e730cece-b817-4691-ab32-96d2e3e8dddc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14e51ac136874806b4b903235697b404'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:18.307469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e730cece-b817-4691-ab32-96d2e3e8dddc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14e51ac136874806b4b903235697b404'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:18.308463 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e730cece-b817-4691-ab32-96d2e3e8dddc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14e51ac136874806b4b903235697b404'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:18.322047 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e730cece-b817-4691-ab32-96d2e3e8dddc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14e51ac136874806b4b903235697b404'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:18.346466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1121/2256] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:05:17 2022] DELETE /identity/v3/users/14e51ac136874806b4b903235697b404 => generated 0 bytes in 873 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:18.967870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fcc2a-3d2a-4a07-9652-74a57f7d2e50 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:19.004638 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2fcc2a-3d2a-4a07-9652-74a57f7d2e50 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:19.105332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58033e-9fe5-4b24-9550-345e9df8501c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:19.105901 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58033e-9fe5-4b24-9550-345e9df8501c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:19.106714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a58033e-9fe5-4b24-9550-345e9df8501c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:19.200231 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1a58033e-9fe5-4b24-9550-345e9df8501c None None] MFA Rules not processed for user `046e50a17a6a4783895782d3aa2172e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:19.203747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1a58033e-9fe5-4b24-9550-345e9df8501c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:19.242613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db5823-27e7-4363-bdfc-e90b49497211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:19.243310 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db5823-27e7-4363-bdfc-e90b49497211 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:19.243686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db5823-27e7-4363-bdfc-e90b49497211 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:19.243950 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db5823-27e7-4363-bdfc-e90b49497211 admin admin] PATH_INFO: `/v3/projects/bcd6fe4cc3554b888bb25819f5fac1e4` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:19.245455 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db5823-27e7-4363-bdfc-e90b49497211 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcd6fe4cc3554b888bb25819f5fac1e4)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:19.248405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db5823-27e7-4363-bdfc-e90b49497211 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:19.340226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1122/2257] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:19.662937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e2db5823-27e7-4363-bdfc-e90b49497211 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcd6fe4cc3554b888bb25819f5fac1e4', 'request_id': 'req-e2db5823-27e7-4363-bdfc-e90b49497211'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:19.670652 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e2db5823-27e7-4363-bdfc-e90b49497211 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcd6fe4cc3554b888bb25819f5fac1e4', 'request_id': 'req-e2db5823-27e7-4363-bdfc-e90b49497211'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:19.875305 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e2db5823-27e7-4363-bdfc-e90b49497211 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcd6fe4cc3554b888bb25819f5fac1e4', 'request_id': 'req-e2db5823-27e7-4363-bdfc-e90b49497211'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:19.895085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e2db5823-27e7-4363-bdfc-e90b49497211 admin admin] The token cache is being invalidate because project bcd6fe4cc3554b888bb25819f5fac1e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:19.895085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e2db5823-27e7-4363-bdfc-e90b49497211 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:19.895085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e2db5823-27e7-4363-bdfc-e90b49497211 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:19.895622 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1136/2258] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:05:18 2022] DELETE /identity/v3/projects/bcd6fe4cc3554b888bb25819f5fac1e4 => generated 0 bytes in 928 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:19.902165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e7eb5-5e57-4816-a07a-131177506b22 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:19.902374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e7eb5-5e57-4816-a07a-131177506b22 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:19.902573 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760e7eb5-5e57-4816-a07a-131177506b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:19.927238 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-760e7eb5-5e57-4816-a07a-131177506b22 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:19.928934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-760e7eb5-5e57-4816-a07a-131177506b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:20.035182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1123/2259] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:05:20.044045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819226e-50f5-4ece-a2b6-b0c62d1724cb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:20.044855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819226e-50f5-4ece-a2b6-b0c62d1724cb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:20.044855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d819226e-50f5-4ece-a2b6-b0c62d1724cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:20.076054 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d819226e-50f5-4ece-a2b6-b0c62d1724cb None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:20.077646 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d819226e-50f5-4ece-a2b6-b0c62d1724cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:20.090509 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1137/2260] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:05:20.095945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e7eb5-5e57-4816-a07a-131177506b22 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:20.106502 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760e7eb5-5e57-4816-a07a-131177506b22 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:20.113088 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b454d-c64f-4060-96f2-5ffb69142c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:20.113797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b454d-c64f-4060-96f2-5ffb69142c28 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:20.114073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b454d-c64f-4060-96f2-5ffb69142c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:20.114625 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b454d-c64f-4060-96f2-5ffb69142c28 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:20.115491 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b454d-c64f-4060-96f2-5ffb69142c28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:05:20.116034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b454d-c64f-4060-96f2-5ffb69142c28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:20.118155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b454d-c64f-4060-96f2-5ffb69142c28 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:20.136132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1124/2261] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:05:20.137599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:20.141195 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819226e-50f5-4ece-a2b6-b0c62d1724cb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:20.155414 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d819226e-50f5-4ece-a2b6-b0c62d1724cb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:20.161511 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8167d-6b62-492e-90ad-ff7523116b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:20.162906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8167d-6b62-492e-90ad-ff7523116b75 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:20.162906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8167d-6b62-492e-90ad-ff7523116b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:20.163234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de8167d-6b62-492e-90ad-ff7523116b75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:20.164923 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8167d-6b62-492e-90ad-ff7523116b75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:20.167513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de8167d-6b62-492e-90ad-ff7523116b75 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:20.228624 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=17,keystone:SELECT=80,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:20.370424 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1138/2262] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:05:20 2022] POST /identity/v3/projects => generated 389 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:20.376538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b454d-c64f-4060-96f2-5ffb69142c28 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:20.394333 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b454d-c64f-4060-96f2-5ffb69142c28 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:20.582882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a76a9-ef02-4359-9bed-58a376bbac8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:20.584973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a76a9-ef02-4359-9bed-58a376bbac8e admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:20.585660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a76a9-ef02-4359-9bed-58a376bbac8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:20.586016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a76a9-ef02-4359-9bed-58a376bbac8e admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:20.587191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a76a9-ef02-4359-9bed-58a376bbac8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:20.591887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a76a9-ef02-4359-9bed-58a376bbac8e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:20.821927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1125/2263] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:05:20 2022] POST /identity/v3/users => generated 295 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:20.827055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8167d-6b62-492e-90ad-ff7523116b75 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:20.841500 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de8167d-6b62-492e-90ad-ff7523116b75 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:20.847958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618d9f4-f2ac-4ca4-8f59-deec34612c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:20.848601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618d9f4-f2ac-4ca4-8f59-deec34612c5b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:20.848905 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618d9f4-f2ac-4ca4-8f59-deec34612c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:20.849159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e618d9f4-f2ac-4ca4-8f59-deec34612c5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:20.851096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618d9f4-f2ac-4ca4-8f59-deec34612c5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:20.861059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e618d9f4-f2ac-4ca4-8f59-deec34612c5b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:20.885191 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1139/2264] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:20.890081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a76a9-ef02-4359-9bed-58a376bbac8e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:20.908753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a76a9-ef02-4359-9bed-58a376bbac8e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:20.916354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace84b0-681a-4d30-950a-9ef0ffae99e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:20.917076 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace84b0-681a-4d30-950a-9ef0ffae99e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:20.917368 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace84b0-681a-4d30-950a-9ef0ffae99e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:20.918007 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace84b0-681a-4d30-950a-9ef0ffae99e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:20.918965 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace84b0-681a-4d30-950a-9ef0ffae99e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:20.921088 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace84b0-681a-4d30-950a-9ef0ffae99e1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:20.941157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1126/2265] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:20.947485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618d9f4-f2ac-4ca4-8f59-deec34612c5b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:20.965862 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e618d9f4-f2ac-4ca4-8f59-deec34612c5b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:20.972002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb95aab-e0b2-4b37-8bfc-c8d542d18149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:20.972970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb95aab-e0b2-4b37-8bfc-c8d542d18149 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:20.973289 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb95aab-e0b2-4b37-8bfc-c8d542d18149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:20.973614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb95aab-e0b2-4b37-8bfc-c8d542d18149 admin admin] PATH_INFO: `/v3/projects/f6595854cfb04a11994c6aed8f5857c6/users/612875c8fc7240e5b18fe38a034a3ef5/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:20.999538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb95aab-e0b2-4b37-8bfc-c8d542d18149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6595854cfb04a11994c6aed8f5857c6, user_id=612875c8fc7240e5b18fe38a034a3ef5, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:21.005183 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb95aab-e0b2-4b37-8bfc-c8d542d18149 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:21.234812 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1140/2266] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:20 2022] PUT /identity/v3/projects/f6595854cfb04a11994c6aed8f5857c6/users/612875c8fc7240e5b18fe38a034a3ef5/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:21.241438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31b771-db76-4ff1-967f-691a2d6b919d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:21.242025 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31b771-db76-4ff1-967f-691a2d6b919d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:21.242386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31b771-db76-4ff1-967f-691a2d6b919d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:21.266044 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6f31b771-db76-4ff1-967f-691a2d6b919d None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:21.269211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6f31b771-db76-4ff1-967f-691a2d6b919d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:21.338839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1127/2267] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:21.348921 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb95aab-e0b2-4b37-8bfc-c8d542d18149 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:21.350700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dfb95aab-e0b2-4b37-8bfc-c8d542d18149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:21.426711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb95aab-e0b2-4b37-8bfc-c8d542d18149 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:21.430698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0151b-920c-4627-9488-be1b6ebe57ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:21.431342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0151b-920c-4627-9488-be1b6ebe57ac service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:21.431637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0151b-920c-4627-9488-be1b6ebe57ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:21.431938 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e0151b-920c-4627-9488-be1b6ebe57ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:21.455084 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0151b-920c-4627-9488-be1b6ebe57ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:21.455084 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e0151b-920c-4627-9488-be1b6ebe57ac service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:21.470001 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1141/2268] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:21.483691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:26.028830 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:27.907510 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:31.627521 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:31.856885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead74bd-8bad-48ad-812b-76cb801e3291 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:31.857328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead74bd-8bad-48ad-812b-76cb801e3291 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:31.857790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ead74bd-8bad-48ad-812b-76cb801e3291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:31.965711 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1ead74bd-8bad-48ad-812b-76cb801e3291 None None] MFA Rules not processed for user `612875c8fc7240e5b18fe38a034a3ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:31.969103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1ead74bd-8bad-48ad-812b-76cb801e3291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:32.172457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1128/2269] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:32.541958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0151b-920c-4627-9488-be1b6ebe57ac service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:32.544081 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e5e0151b-920c-4627-9488-be1b6ebe57ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:32.663969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e0151b-920c-4627-9488-be1b6ebe57ac service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:32.672504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8f9adf-0ccf-434e-9b8f-311e7ecb3d7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:32.675324 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9adf-0ccf-434e-9b8f-311e7ecb3d7c service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:32.675948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9adf-0ccf-434e-9b8f-311e7ecb3d7c service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:32.676407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8f9adf-0ccf-434e-9b8f-311e7ecb3d7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:32.688750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f9adf-0ccf-434e-9b8f-311e7ecb3d7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:32.692213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8f9adf-0ccf-434e-9b8f-311e7ecb3d7c service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:32.708509 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1142/2270] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:33.271048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:33.811400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead74bd-8bad-48ad-812b-76cb801e3291 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:33.815217 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1ead74bd-8bad-48ad-812b-76cb801e3291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:33.839443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ead74bd-8bad-48ad-812b-76cb801e3291 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:33.850012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ea6ad-fbfa-4dc8-bfa1-15226beebb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:33.851280 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ea6ad-fbfa-4dc8-bfa1-15226beebb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:33.851844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ea6ad-fbfa-4dc8-bfa1-15226beebb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:33.852385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ea6ad-fbfa-4dc8-bfa1-15226beebb9f admin admin] PATH_INFO: `/v3/users/a50100444983412180bb2b34bfefd7e2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:33.854917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ea6ad-fbfa-4dc8-bfa1-15226beebb9f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a50100444983412180bb2b34bfefd7e2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:33.858623 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ea6ad-fbfa-4dc8-bfa1-15226beebb9f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:34.170137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-527ea6ad-fbfa-4dc8-bfa1-15226beebb9f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a50100444983412180bb2b34bfefd7e2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:34.248007 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-527ea6ad-fbfa-4dc8-bfa1-15226beebb9f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a50100444983412180bb2b34bfefd7e2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:34.249526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-527ea6ad-fbfa-4dc8-bfa1-15226beebb9f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a50100444983412180bb2b34bfefd7e2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:34.250925 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-527ea6ad-fbfa-4dc8-bfa1-15226beebb9f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a50100444983412180bb2b34bfefd7e2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:34.265466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-527ea6ad-fbfa-4dc8-bfa1-15226beebb9f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a50100444983412180bb2b34bfefd7e2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:34.289002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1129/2271] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:05:33 2022] DELETE /identity/v3/users/a50100444983412180bb2b34bfefd7e2 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:34.294297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9fe09-76d2-4a38-861a-1acda3f3cf20 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:34.295125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9fe09-76d2-4a38-861a-1acda3f3cf20 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:34.295395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9fe09-76d2-4a38-861a-1acda3f3cf20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:34.321129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f6c9fe09-76d2-4a38-861a-1acda3f3cf20 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:34.322402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f6c9fe09-76d2-4a38-861a-1acda3f3cf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:34.405145 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1143/2272] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:05:34.418691 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ea6ad-fbfa-4dc8-bfa1-15226beebb9f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:34.420274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-527ea6ad-fbfa-4dc8-bfa1-15226beebb9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:34.487181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ea6ad-fbfa-4dc8-bfa1-15226beebb9f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:34.491535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f7748-c652-4839-860b-81220f9857f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:34.492434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f7748-c652-4839-860b-81220f9857f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:34.498077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f7748-c652-4839-860b-81220f9857f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:34.498380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f7748-c652-4839-860b-81220f9857f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:34.511506 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f7748-c652-4839-860b-81220f9857f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:34.513226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f7748-c652-4839-860b-81220f9857f2 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:34.526583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1130/2273] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:35.238752 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9fe09-76d2-4a38-861a-1acda3f3cf20 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:35.239900 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f6c9fe09-76d2-4a38-861a-1acda3f3cf20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:35.261728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9fe09-76d2-4a38-861a-1acda3f3cf20 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:35.283433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41166a9-b7a0-4aac-8ed2-c9c509d721d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:35.284155 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41166a9-b7a0-4aac-8ed2-c9c509d721d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:35.284445 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41166a9-b7a0-4aac-8ed2-c9c509d721d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:35.284723 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41166a9-b7a0-4aac-8ed2-c9c509d721d7 admin admin] PATH_INFO: `/v3/projects/f01b46a2d7c94d24b22baffd180902a4` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:35.287419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41166a9-b7a0-4aac-8ed2-c9c509d721d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f01b46a2d7c94d24b22baffd180902a4)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:35.297753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41166a9-b7a0-4aac-8ed2-c9c509d721d7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:35.573194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b41166a9-b7a0-4aac-8ed2-c9c509d721d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f01b46a2d7c94d24b22baffd180902a4', 'request_id': 'req-b41166a9-b7a0-4aac-8ed2-c9c509d721d7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:35.582467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b41166a9-b7a0-4aac-8ed2-c9c509d721d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f01b46a2d7c94d24b22baffd180902a4', 'request_id': 'req-b41166a9-b7a0-4aac-8ed2-c9c509d721d7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:35.740455 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b41166a9-b7a0-4aac-8ed2-c9c509d721d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f01b46a2d7c94d24b22baffd180902a4', 'request_id': 'req-b41166a9-b7a0-4aac-8ed2-c9c509d721d7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:35.754289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:35.789443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b41166a9-b7a0-4aac-8ed2-c9c509d721d7 admin admin] The token cache is being invalidate because project f01b46a2d7c94d24b22baffd180902a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:35.789682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b41166a9-b7a0-4aac-8ed2-c9c509d721d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:35.790769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b41166a9-b7a0-4aac-8ed2-c9c509d721d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:35.793733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1144/2274] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:05:35 2022] DELETE /identity/v3/projects/f01b46a2d7c94d24b22baffd180902a4 => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:35.799626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f7748-c652-4839-860b-81220f9857f2 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:35.816009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f7748-c652-4839-860b-81220f9857f2 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:35.942982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0361c13-2400-47ae-aa16-362722ed35d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:35.944191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0361c13-2400-47ae-aa16-362722ed35d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:35.944852 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0361c13-2400-47ae-aa16-362722ed35d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:35.945390 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0361c13-2400-47ae-aa16-362722ed35d5 admin admin] PATH_INFO: `/v3/users/8cec29c4515345a88fea1795fdaa4d49` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:35.953293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0361c13-2400-47ae-aa16-362722ed35d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cec29c4515345a88fea1795fdaa4d49)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:35.957523 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0361c13-2400-47ae-aa16-362722ed35d5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:36.398409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b0361c13-2400-47ae-aa16-362722ed35d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cec29c4515345a88fea1795fdaa4d49'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:36.646409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b0361c13-2400-47ae-aa16-362722ed35d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cec29c4515345a88fea1795fdaa4d49'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:36.647070 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b0361c13-2400-47ae-aa16-362722ed35d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cec29c4515345a88fea1795fdaa4d49'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:36.647700 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b0361c13-2400-47ae-aa16-362722ed35d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cec29c4515345a88fea1795fdaa4d49'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:36.662640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b0361c13-2400-47ae-aa16-362722ed35d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cec29c4515345a88fea1795fdaa4d49'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:36.699694 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1131/2275] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:05:35 2022] DELETE /identity/v3/users/8cec29c4515345a88fea1795fdaa4d49 => generated 0 bytes in 901 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:37.262039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de892a5a-c7db-4128-9f3d-923c9cc8f607 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:37.262858 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de892a5a-c7db-4128-9f3d-923c9cc8f607 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:37.263620 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de892a5a-c7db-4128-9f3d-923c9cc8f607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:37.318093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-de892a5a-c7db-4128-9f3d-923c9cc8f607 None None] MFA Rules not processed for user `612875c8fc7240e5b18fe38a034a3ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:37.318093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-de892a5a-c7db-4128-9f3d-923c9cc8f607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:37.420425 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1145/2276] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:37.565741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0361c13-2400-47ae-aa16-362722ed35d5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:37.567422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b0361c13-2400-47ae-aa16-362722ed35d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:37.705060 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0361c13-2400-47ae-aa16-362722ed35d5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:37.710663 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e84c3-73d8-462f-874e-1cc36c502709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:37.712104 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e84c3-73d8-462f-874e-1cc36c502709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:37.712513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e84c3-73d8-462f-874e-1cc36c502709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:37.712939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e84c3-73d8-462f-874e-1cc36c502709 admin admin] PATH_INFO: `/v3/projects/ac4bc07f889d4394a9ac2ffdea6d8582` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:37.722145 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e84c3-73d8-462f-874e-1cc36c502709 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac4bc07f889d4394a9ac2ffdea6d8582)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:37.730250 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e84c3-73d8-462f-874e-1cc36c502709 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:37.911533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae4e84c3-73d8-462f-874e-1cc36c502709 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac4bc07f889d4394a9ac2ffdea6d8582', 'request_id': 'req-ae4e84c3-73d8-462f-874e-1cc36c502709'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:37.920986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae4e84c3-73d8-462f-874e-1cc36c502709 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac4bc07f889d4394a9ac2ffdea6d8582', 'request_id': 'req-ae4e84c3-73d8-462f-874e-1cc36c502709'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:37.996704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae4e84c3-73d8-462f-874e-1cc36c502709 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac4bc07f889d4394a9ac2ffdea6d8582', 'request_id': 'req-ae4e84c3-73d8-462f-874e-1cc36c502709'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:38.035940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae4e84c3-73d8-462f-874e-1cc36c502709 admin admin] The token cache is being invalidate because project ac4bc07f889d4394a9ac2ffdea6d8582 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:38.037834 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae4e84c3-73d8-462f-874e-1cc36c502709 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:38.040293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae4e84c3-73d8-462f-874e-1cc36c502709 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:38.048785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1132/2277] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:05:37 2022] DELETE /identity/v3/projects/ac4bc07f889d4394a9ac2ffdea6d8582 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:38.060748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987c57e-e4e4-49d9-a087-f1569ae8ca1b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:38.061606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987c57e-e4e4-49d9-a087-f1569ae8ca1b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:38.062952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987c57e-e4e4-49d9-a087-f1569ae8ca1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:38.101476 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b987c57e-e4e4-49d9-a087-f1569ae8ca1b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:38.102825 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b987c57e-e4e4-49d9-a087-f1569ae8ca1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:38.251518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1146/2278] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:38.251845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83b9cc-209d-4c4d-b781-d5b607bf4a90 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:38.252159 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83b9cc-209d-4c4d-b781-d5b607bf4a90 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:38.252769 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83b9cc-209d-4c4d-b781-d5b607bf4a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:38.283746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3f83b9cc-209d-4c4d-b781-d5b607bf4a90 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:38.283746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3f83b9cc-209d-4c4d-b781-d5b607bf4a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:38.293967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1133/2279] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:38.298415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987c57e-e4e4-49d9-a087-f1569ae8ca1b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:38.308396 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987c57e-e4e4-49d9-a087-f1569ae8ca1b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:38.317711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76a3a2-e2f0-4b29-8d7a-8b63bbe93cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:38.318489 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76a3a2-e2f0-4b29-8d7a-8b63bbe93cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:38.320782 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76a3a2-e2f0-4b29-8d7a-8b63bbe93cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:38.320782 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76a3a2-e2f0-4b29-8d7a-8b63bbe93cf1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:38.320782 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76a3a2-e2f0-4b29-8d7a-8b63bbe93cf1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:05:38.320782 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76a3a2-e2f0-4b29-8d7a-8b63bbe93cf1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:38.322157 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff76a3a2-e2f0-4b29-8d7a-8b63bbe93cf1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:38.339167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1147/2280] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:05:38.350906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83b9cc-209d-4c4d-b781-d5b607bf4a90 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:38.369005 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83b9cc-209d-4c4d-b781-d5b607bf4a90 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:38.376617 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df65d99-6c0d-4ffb-b348-9666e9f40a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:38.377493 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df65d99-6c0d-4ffb-b348-9666e9f40a08 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:38.377923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df65d99-6c0d-4ffb-b348-9666e9f40a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:38.378222 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df65d99-6c0d-4ffb-b348-9666e9f40a08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:38.378868 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df65d99-6c0d-4ffb-b348-9666e9f40a08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:38.385178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df65d99-6c0d-4ffb-b348-9666e9f40a08 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:38.607638 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1134/2281] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:05:38 2022] POST /identity/v3/projects => generated 363 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:38.611675 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76a3a2-e2f0-4b29-8d7a-8b63bbe93cf1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:38.622868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff76a3a2-e2f0-4b29-8d7a-8b63bbe93cf1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:38.715267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2ce00-f72f-47a3-bd72-382db518925c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:38.716039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2ce00-f72f-47a3-bd72-382db518925c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:38.716399 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2ce00-f72f-47a3-bd72-382db518925c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:38.716722 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd2ce00-f72f-47a3-bd72-382db518925c admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:38.718200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd2ce00-f72f-47a3-bd72-382db518925c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:38.722315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd2ce00-f72f-47a3-bd72-382db518925c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:38.996051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1148/2282] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:05:38 2022] POST /identity/v3/users => generated 282 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:05:39.000701 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df65d99-6c0d-4ffb-b348-9666e9f40a08 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:39.023761 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df65d99-6c0d-4ffb-b348-9666e9f40a08 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:39.025591 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31cc19-e70b-4b6b-8189-2b41c1e14eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:39.027172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31cc19-e70b-4b6b-8189-2b41c1e14eae admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:39.027172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31cc19-e70b-4b6b-8189-2b41c1e14eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:39.027761 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e31cc19-e70b-4b6b-8189-2b41c1e14eae admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:39.029143 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31cc19-e70b-4b6b-8189-2b41c1e14eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:39.032926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e31cc19-e70b-4b6b-8189-2b41c1e14eae admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:39.046149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1135/2283] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:39.055703 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2ce00-f72f-47a3-bd72-382db518925c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:39.086367 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd2ce00-f72f-47a3-bd72-382db518925c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:39.092956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595a1f3-fb0d-4730-9bc1-92b8eee52add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:39.099308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595a1f3-fb0d-4730-9bc1-92b8eee52add admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:39.099308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595a1f3-fb0d-4730-9bc1-92b8eee52add admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:39.099308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595a1f3-fb0d-4730-9bc1-92b8eee52add admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:39.099308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595a1f3-fb0d-4730-9bc1-92b8eee52add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:39.099308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595a1f3-fb0d-4730-9bc1-92b8eee52add admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:39.121025 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1149/2284] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:39.126165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31cc19-e70b-4b6b-8189-2b41c1e14eae admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:39.143067 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e31cc19-e70b-4b6b-8189-2b41c1e14eae admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:39.149344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc46f1-ab2b-4ce7-80f1-9e6429193da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:39.150162 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc46f1-ab2b-4ce7-80f1-9e6429193da8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:39.150513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc46f1-ab2b-4ce7-80f1-9e6429193da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:39.150897 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc46f1-ab2b-4ce7-80f1-9e6429193da8 admin admin] PATH_INFO: `/v3/projects/a0495e2abcab48aabdf623183f87717e/users/b286c8793c9546428c04e3c69b21bec3/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:39.180693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc46f1-ab2b-4ce7-80f1-9e6429193da8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0495e2abcab48aabdf623183f87717e, user_id=b286c8793c9546428c04e3c69b21bec3, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:39.183801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc46f1-ab2b-4ce7-80f1-9e6429193da8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:39.387447 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1136/2285] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:39 2022] PUT /identity/v3/projects/a0495e2abcab48aabdf623183f87717e/users/b286c8793c9546428c04e3c69b21bec3/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:39.400815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331863c-0138-4f86-8805-a0b67b9181b7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:39.401428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331863c-0138-4f86-8805-a0b67b9181b7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:39.401999 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331863c-0138-4f86-8805-a0b67b9181b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:39.470179 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1331863c-0138-4f86-8805-a0b67b9181b7 None None] MFA Rules not processed for user `b286c8793c9546428c04e3c69b21bec3`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:39.471357 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1331863c-0138-4f86-8805-a0b67b9181b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:39.591925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1150/2286] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:05:39.599826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc46f1-ab2b-4ce7-80f1-9e6429193da8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:39.690963 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc46f1-ab2b-4ce7-80f1-9e6429193da8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:39.696133 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74522b8d-43d9-4460-8dbe-53b7f32888a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:39.696932 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74522b8d-43d9-4460-8dbe-53b7f32888a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:39.697290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74522b8d-43d9-4460-8dbe-53b7f32888a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:39.697643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74522b8d-43d9-4460-8dbe-53b7f32888a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:39.698400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74522b8d-43d9-4460-8dbe-53b7f32888a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:39.700493 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74522b8d-43d9-4460-8dbe-53b7f32888a9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:39.861061 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1137/2287] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:05:39 2022] POST /identity/v3/projects => generated 365 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:39.865585 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331863c-0138-4f86-8805-a0b67b9181b7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:39.877719 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1331863c-0138-4f86-8805-a0b67b9181b7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:39.945138 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afde2c-ca8f-4a5d-9b30-5741cecf006d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:39.945762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afde2c-ca8f-4a5d-9b30-5741cecf006d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:39.946051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afde2c-ca8f-4a5d-9b30-5741cecf006d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:39.946329 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41afde2c-ca8f-4a5d-9b30-5741cecf006d admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:39.946987 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41afde2c-ca8f-4a5d-9b30-5741cecf006d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:39.948993 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41afde2c-ca8f-4a5d-9b30-5741cecf006d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:40.104735 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1151/2288] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:05:39 2022] POST /identity/v3/users => generated 289 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:40.108870 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74522b8d-43d9-4460-8dbe-53b7f32888a9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:40.120725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74522b8d-43d9-4460-8dbe-53b7f32888a9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:40.126785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c3f0f-af86-473c-8897-1aceff5d8fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:40.128519 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c3f0f-af86-473c-8897-1aceff5d8fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:40.129039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c3f0f-af86-473c-8897-1aceff5d8fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:40.129399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c3f0f-af86-473c-8897-1aceff5d8fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:40.130629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c3f0f-af86-473c-8897-1aceff5d8fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:40.132817 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c3f0f-af86-473c-8897-1aceff5d8fd8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:40.146020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1138/2289] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:05:40.150429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afde2c-ca8f-4a5d-9b30-5741cecf006d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:40.164152 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41afde2c-ca8f-4a5d-9b30-5741cecf006d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:40.171990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963df528-4cdb-4a29-a6a4-f2b87386fe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:40.173584 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963df528-4cdb-4a29-a6a4-f2b87386fe5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:40.174057 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963df528-4cdb-4a29-a6a4-f2b87386fe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:40.174506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963df528-4cdb-4a29-a6a4-f2b87386fe5a admin admin] PATH_INFO: `/v3/projects/e2ceb78d4f2843879f55a6c9093b78f7/users/99529005814945cbb4a0030175743b36/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:40.195403 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963df528-4cdb-4a29-a6a4-f2b87386fe5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2ceb78d4f2843879f55a6c9093b78f7, user_id=99529005814945cbb4a0030175743b36, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:40.198965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963df528-4cdb-4a29-a6a4-f2b87386fe5a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:40.448224 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1152/2290] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:40 2022] PUT /identity/v3/projects/e2ceb78d4f2843879f55a6c9093b78f7/users/99529005814945cbb4a0030175743b36/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:40.452286 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c3f0f-af86-473c-8897-1aceff5d8fd8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:40.464664 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c3f0f-af86-473c-8897-1aceff5d8fd8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:40.488843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963df528-4cdb-4a29-a6a4-f2b87386fe5a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:40.490801 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-963df528-4cdb-4a29-a6a4-f2b87386fe5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:40.551320 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7dcf30-bddd-417d-8826-7732e4b3242a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:40.551879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dcf30-bddd-417d-8826-7732e4b3242a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:40.552164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dcf30-bddd-417d-8826-7732e4b3242a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:40.552435 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7dcf30-bddd-417d-8826-7732e4b3242a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:40.553093 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7dcf30-bddd-417d-8826-7732e4b3242a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:40.555080 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7dcf30-bddd-417d-8826-7732e4b3242a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:40.576749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1139/2291] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:40 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:40.583581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7dcf30-bddd-417d-8826-7732e4b3242a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:40.624028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963df528-4cdb-4a29-a6a4-f2b87386fe5a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:40.633959 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7dcf30-bddd-417d-8826-7732e4b3242a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:40.664201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba331f1f-dd9b-4daa-a6bc-4ca6f5eeeeae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:40.664746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e5ee5-9f8a-42fe-b024-52cacacce5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:40.665944 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e5ee5-9f8a-42fe-b024-52cacacce5d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:40.666507 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e5ee5-9f8a-42fe-b024-52cacacce5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:40.667138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e5ee5-9f8a-42fe-b024-52cacacce5d8 admin admin] PATH_INFO: `/v3/projects/e2ceb78d4f2843879f55a6c9093b78f7/users/99529005814945cbb4a0030175743b36/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:40.667832 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba331f1f-dd9b-4daa-a6bc-4ca6f5eeeeae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:40.668388 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba331f1f-dd9b-4daa-a6bc-4ca6f5eeeeae admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:40.668854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba331f1f-dd9b-4daa-a6bc-4ca6f5eeeeae admin admin] PATH_INFO: `/v3/users/48526370b77345a4b590ab7b84bf0837` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:40.673281 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba331f1f-dd9b-4daa-a6bc-4ca6f5eeeeae admin admin] RBAC: Authorizing `identity:delete_user(user_id=48526370b77345a4b590ab7b84bf0837)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:40.678718 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e5ee5-9f8a-42fe-b024-52cacacce5d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2ceb78d4f2843879f55a6c9093b78f7, user_id=99529005814945cbb4a0030175743b36, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:40.679549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba331f1f-dd9b-4daa-a6bc-4ca6f5eeeeae admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:40.694660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e5ee5-9f8a-42fe-b024-52cacacce5d8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:40.733658 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1140/2292] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:40 2022] PUT /identity/v3/projects/e2ceb78d4f2843879f55a6c9093b78f7/users/99529005814945cbb4a0030175743b36/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:05:40.890255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb490f-daee-4fa4-ae41-a5ab6b0df3f5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:40.890255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb490f-daee-4fa4-ae41-a5ab6b0df3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:40.890255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffb490f-daee-4fa4-ae41-a5ab6b0df3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:40.921209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fffb490f-daee-4fa4-ae41-a5ab6b0df3f5 None None] MFA Rules not processed for user `99529005814945cbb4a0030175743b36`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:40.922483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fffb490f-daee-4fa4-ae41-a5ab6b0df3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:41.021459 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1141/2293] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:05:41.033203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb490f-daee-4fa4-ae41-a5ab6b0df3f5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:41.034946 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fffb490f-daee-4fa4-ae41-a5ab6b0df3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:41.097091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffb490f-daee-4fa4-ae41-a5ab6b0df3f5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:41.101517 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee9c10-1ffd-4eb3-99fb-6da6a5d0eedd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:41.102442 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee9c10-1ffd-4eb3-99fb-6da6a5d0eedd service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:41.102848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee9c10-1ffd-4eb3-99fb-6da6a5d0eedd service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:41.103505 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ee9c10-1ffd-4eb3-99fb-6da6a5d0eedd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:41.114845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee9c10-1ffd-4eb3-99fb-6da6a5d0eedd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:41.118187 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ee9c10-1ffd-4eb3-99fb-6da6a5d0eedd service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:41.137495 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1142/2294] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:05:41.820126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:41.927354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee9c10-1ffd-4eb3-99fb-6da6a5d0eedd service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:41.940854 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ee9c10-1ffd-4eb3-99fb-6da6a5d0eedd service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:41.941513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ba331f1f-dd9b-4daa-a6bc-4ca6f5eeeeae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48526370b77345a4b590ab7b84bf0837'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:41.946996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb3e52-c309-41d1-9354-08d16ca7e561 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:41.948341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb3e52-c309-41d1-9354-08d16ca7e561 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:41.949147 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb3e52-c309-41d1-9354-08d16ca7e561 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:41.949827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb3e52-c309-41d1-9354-08d16ca7e561 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:42.050165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb3e52-c309-41d1-9354-08d16ca7e561 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:42.051756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb3e52-c309-41d1-9354-08d16ca7e561 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:42.064357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1143/2295] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:41 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:42.145463 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:42.201479 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ba331f1f-dd9b-4daa-a6bc-4ca6f5eeeeae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48526370b77345a4b590ab7b84bf0837'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:42.202349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ba331f1f-dd9b-4daa-a6bc-4ca6f5eeeeae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48526370b77345a4b590ab7b84bf0837'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:42.203310 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ba331f1f-dd9b-4daa-a6bc-4ca6f5eeeeae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48526370b77345a4b590ab7b84bf0837'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:42.218484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ba331f1f-dd9b-4daa-a6bc-4ca6f5eeeeae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48526370b77345a4b590ab7b84bf0837'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:42.235010 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1153/2296] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:05:40 2022] DELETE /identity/v3/users/48526370b77345a4b590ab7b84bf0837 => generated 0 bytes in 1747 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:42.241034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa7637-a5b8-49da-9574-0e4cb3a1792a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:42.241403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa7637-a5b8-49da-9574-0e4cb3a1792a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:42.241749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fa7637-a5b8-49da-9574-0e4cb3a1792a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:42.264650 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-10fa7637-a5b8-49da-9574-0e4cb3a1792a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:42.266192 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-10fa7637-a5b8-49da-9574-0e4cb3a1792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:42.343273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1144/2297] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:42.354309 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba331f1f-dd9b-4daa-a6bc-4ca6f5eeeeae admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:42.355860 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ba331f1f-dd9b-4daa-a6bc-4ca6f5eeeeae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:42.429931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba331f1f-dd9b-4daa-a6bc-4ca6f5eeeeae admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:42.433946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59254f-f636-4c0f-aed2-2a9afdec01cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:42.434629 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59254f-f636-4c0f-aed2-2a9afdec01cf service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:42.434986 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59254f-f636-4c0f-aed2-2a9afdec01cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:42.435263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59254f-f636-4c0f-aed2-2a9afdec01cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:42.445785 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59254f-f636-4c0f-aed2-2a9afdec01cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:42.447283 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59254f-f636-4c0f-aed2-2a9afdec01cf service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:42.459299 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1154/2298] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:43.427287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa7637-a5b8-49da-9574-0e4cb3a1792a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:43.429107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-10fa7637-a5b8-49da-9574-0e4cb3a1792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:43.443603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fa7637-a5b8-49da-9574-0e4cb3a1792a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:43.448881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76103a0d-d8d3-4f6d-a0a6-ad23e77c2d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:43.449525 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76103a0d-d8d3-4f6d-a0a6-ad23e77c2d2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:43.449913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76103a0d-d8d3-4f6d-a0a6-ad23e77c2d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:43.450191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76103a0d-d8d3-4f6d-a0a6-ad23e77c2d2c admin admin] PATH_INFO: `/v3/projects/f384fc2a628a48c6b46d021f4dd42beb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:43.451388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76103a0d-d8d3-4f6d-a0a6-ad23e77c2d2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f384fc2a628a48c6b46d021f4dd42beb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:43.453495 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76103a0d-d8d3-4f6d-a0a6-ad23e77c2d2c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:43.642872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-76103a0d-d8d3-4f6d-a0a6-ad23e77c2d2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f384fc2a628a48c6b46d021f4dd42beb', 'request_id': 'req-76103a0d-d8d3-4f6d-a0a6-ad23e77c2d2c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:43.649448 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-76103a0d-d8d3-4f6d-a0a6-ad23e77c2d2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f384fc2a628a48c6b46d021f4dd42beb', 'request_id': 'req-76103a0d-d8d3-4f6d-a0a6-ad23e77c2d2c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:43.770347 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-76103a0d-d8d3-4f6d-a0a6-ad23e77c2d2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f384fc2a628a48c6b46d021f4dd42beb', 'request_id': 'req-76103a0d-d8d3-4f6d-a0a6-ad23e77c2d2c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:43.788380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-76103a0d-d8d3-4f6d-a0a6-ad23e77c2d2c admin admin] The token cache is being invalidate because project f384fc2a628a48c6b46d021f4dd42beb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:43.788915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-76103a0d-d8d3-4f6d-a0a6-ad23e77c2d2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:43.790298 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-76103a0d-d8d3-4f6d-a0a6-ad23e77c2d2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:43.792730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1145/2299] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:05:43 2022] DELETE /identity/v3/projects/f384fc2a628a48c6b46d021f4dd42beb => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:43.799097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1b760-5b08-4a3e-bb0d-6cf5ff5834ce None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:43.799705 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1b760-5b08-4a3e-bb0d-6cf5ff5834ce None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:43.800037 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1b760-5b08-4a3e-bb0d-6cf5ff5834ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:43.832342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-90a1b760-5b08-4a3e-bb0d-6cf5ff5834ce None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:43.835178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-90a1b760-5b08-4a3e-bb0d-6cf5ff5834ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:43.921937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1155/2300] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:43.936801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f709a-2ef3-4c12-917f-4f12dc5d783a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:43.937257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f709a-2ef3-4c12-917f-4f12dc5d783a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:43.937736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f709a-2ef3-4c12-917f-4f12dc5d783a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:43.973526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-315f709a-2ef3-4c12-917f-4f12dc5d783a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:43.975951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-315f709a-2ef3-4c12-917f-4f12dc5d783a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:44.000036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1146/2301] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:44.019201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1b760-5b08-4a3e-bb0d-6cf5ff5834ce None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:44.045980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1b760-5b08-4a3e-bb0d-6cf5ff5834ce None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:44.061622 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac62b1-b888-4f51-9a6c-b81db090beb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:44.062313 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac62b1-b888-4f51-9a6c-b81db090beb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:44.063221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac62b1-b888-4f51-9a6c-b81db090beb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:44.064686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac62b1-b888-4f51-9a6c-b81db090beb0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:44.065439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac62b1-b888-4f51-9a6c-b81db090beb0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:05:44.065757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac62b1-b888-4f51-9a6c-b81db090beb0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:44.067981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac62b1-b888-4f51-9a6c-b81db090beb0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:44.115464 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1156/2302] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:05:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:05:44.120478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f709a-2ef3-4c12-917f-4f12dc5d783a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:44.134603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f709a-2ef3-4c12-917f-4f12dc5d783a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:44.139707 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fbdff-bd4b-4787-878c-32d0bb71bb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:44.140331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fbdff-bd4b-4787-878c-32d0bb71bb9f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:44.140759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fbdff-bd4b-4787-878c-32d0bb71bb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:44.140759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fbdff-bd4b-4787-878c-32d0bb71bb9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:44.142358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fbdff-bd4b-4787-878c-32d0bb71bb9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:44.144603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fbdff-bd4b-4787-878c-32d0bb71bb9f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:44.277317 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1147/2303] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:05:44 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:44.285116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac62b1-b888-4f51-9a6c-b81db090beb0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:44.296706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac62b1-b888-4f51-9a6c-b81db090beb0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:44.369549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fb6f2-07fe-4a64-a7f6-992a9d3a5717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:44.370311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fb6f2-07fe-4a64-a7f6-992a9d3a5717 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:44.370631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fb6f2-07fe-4a64-a7f6-992a9d3a5717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:44.370913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fb6f2-07fe-4a64-a7f6-992a9d3a5717 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:44.371577 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fb6f2-07fe-4a64-a7f6-992a9d3a5717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:44.373546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fb6f2-07fe-4a64-a7f6-992a9d3a5717 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:44.535733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1157/2304] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:05:44 2022] POST /identity/v3/users => generated 292 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:44.538355 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fbdff-bd4b-4787-878c-32d0bb71bb9f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:44.553092 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fbdff-bd4b-4787-878c-32d0bb71bb9f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:44.558406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09506c8-a3ef-41b2-8662-bebbdf8470cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:44.559042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09506c8-a3ef-41b2-8662-bebbdf8470cc admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:44.559323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09506c8-a3ef-41b2-8662-bebbdf8470cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:44.559675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09506c8-a3ef-41b2-8662-bebbdf8470cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:44.560641 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09506c8-a3ef-41b2-8662-bebbdf8470cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:44.563861 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09506c8-a3ef-41b2-8662-bebbdf8470cc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:44.577168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1148/2305] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:44.587317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fb6f2-07fe-4a64-a7f6-992a9d3a5717 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:44.599338 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fb6f2-07fe-4a64-a7f6-992a9d3a5717 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:44.619682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cee384-16da-46d1-b5c7-38709395a8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:44.619682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cee384-16da-46d1-b5c7-38709395a8ea admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:44.620063 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cee384-16da-46d1-b5c7-38709395a8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:44.620157 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cee384-16da-46d1-b5c7-38709395a8ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:44.621427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cee384-16da-46d1-b5c7-38709395a8ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:44.623077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1cee384-16da-46d1-b5c7-38709395a8ea admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:44.637087 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1158/2306] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:44.643285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09506c8-a3ef-41b2-8662-bebbdf8470cc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:44.676194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09506c8-a3ef-41b2-8662-bebbdf8470cc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:44.682412 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74059d9a-6a68-4883-a24b-d2f6f98095cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:44.683149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74059d9a-6a68-4883-a24b-d2f6f98095cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:44.683481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74059d9a-6a68-4883-a24b-d2f6f98095cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:44.683890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74059d9a-6a68-4883-a24b-d2f6f98095cf admin admin] PATH_INFO: `/v3/projects/71ab740a9b3b458e9bb33bbbe4ad24f4/users/b74b92ce24d34b7e9db2c8394d458909/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:44.706486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74059d9a-6a68-4883-a24b-d2f6f98095cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=71ab740a9b3b458e9bb33bbbe4ad24f4, user_id=b74b92ce24d34b7e9db2c8394d458909, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:44.712360 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74059d9a-6a68-4883-a24b-d2f6f98095cf admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:44.873687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1149/2307] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:44 2022] PUT /identity/v3/projects/71ab740a9b3b458e9bb33bbbe4ad24f4/users/b74b92ce24d34b7e9db2c8394d458909/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:05:44.875787 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510e448-6c29-4fbe-a06e-b28be6c7b9a2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:44.877713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510e448-6c29-4fbe-a06e-b28be6c7b9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:44.878183 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510e448-6c29-4fbe-a06e-b28be6c7b9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:44.935627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d510e448-6c29-4fbe-a06e-b28be6c7b9a2 None None] MFA Rules not processed for user `b74b92ce24d34b7e9db2c8394d458909`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:44.939284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d510e448-6c29-4fbe-a06e-b28be6c7b9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:45.097048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1159/2308] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:05:45.117323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74059d9a-6a68-4883-a24b-d2f6f98095cf admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:45.119468 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-74059d9a-6a68-4883-a24b-d2f6f98095cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:45.277813 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74059d9a-6a68-4883-a24b-d2f6f98095cf admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:45.284903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4811c60-89d9-4f91-b7c0-925bb00ab98c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:45.285810 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4811c60-89d9-4f91-b7c0-925bb00ab98c service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:45.286241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4811c60-89d9-4f91-b7c0-925bb00ab98c service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:45.287580 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4811c60-89d9-4f91-b7c0-925bb00ab98c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:45.303345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4811c60-89d9-4f91-b7c0-925bb00ab98c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:45.311315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4811c60-89d9-4f91-b7c0-925bb00ab98c service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:45.349007 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1150/2309] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:45 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:46.128831 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:46.367533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5c38c-2fbe-467b-84e3-9255f97efd28 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:46.368229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5c38c-2fbe-467b-84e3-9255f97efd28 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:46.368863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5c38c-2fbe-467b-84e3-9255f97efd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:46.385385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:46.434060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-eee5c38c-2fbe-467b-84e3-9255f97efd28 None None] MFA Rules not processed for user `0bede3a601f941e7abc9fd16983679b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:46.436303 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-eee5c38c-2fbe-467b-84e3-9255f97efd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:46.551875 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1160/2310] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:46.659754 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4811c60-89d9-4f91-b7c0-925bb00ab98c service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:46.662895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b4811c60-89d9-4f91-b7c0-925bb00ab98c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:46.845100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4811c60-89d9-4f91-b7c0-925bb00ab98c service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:46.856572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad47ff-af79-4949-85c9-15fd9c8f1682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:46.857725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad47ff-af79-4949-85c9-15fd9c8f1682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:46.858349 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad47ff-af79-4949-85c9-15fd9c8f1682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:46.858671 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdad47ff-af79-4949-85c9-15fd9c8f1682 admin admin] PATH_INFO: `/v3/users/b286c8793c9546428c04e3c69b21bec3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:46.861005 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad47ff-af79-4949-85c9-15fd9c8f1682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b286c8793c9546428c04e3c69b21bec3)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:46.865125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdad47ff-af79-4949-85c9-15fd9c8f1682 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:47.127805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fdad47ff-af79-4949-85c9-15fd9c8f1682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b286c8793c9546428c04e3c69b21bec3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:47.338396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fdad47ff-af79-4949-85c9-15fd9c8f1682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b286c8793c9546428c04e3c69b21bec3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:47.339594 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fdad47ff-af79-4949-85c9-15fd9c8f1682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b286c8793c9546428c04e3c69b21bec3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:47.340787 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fdad47ff-af79-4949-85c9-15fd9c8f1682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b286c8793c9546428c04e3c69b21bec3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:47.356112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fdad47ff-af79-4949-85c9-15fd9c8f1682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b286c8793c9546428c04e3c69b21bec3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:47.387914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1151/2311] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:05:46 2022] DELETE /identity/v3/users/b286c8793c9546428c04e3c69b21bec3 => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:47.397075 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e235054-090a-4a19-ac54-1418d1cbca2a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:47.397075 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e235054-090a-4a19-ac54-1418d1cbca2a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:47.397075 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e235054-090a-4a19-ac54-1418d1cbca2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:47.424619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8e235054-090a-4a19-ac54-1418d1cbca2a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:47.426492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8e235054-090a-4a19-ac54-1418d1cbca2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:47.513999 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1161/2312] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:05:47.528040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad47ff-af79-4949-85c9-15fd9c8f1682 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:47.534236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fdad47ff-af79-4949-85c9-15fd9c8f1682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:47.607577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdad47ff-af79-4949-85c9-15fd9c8f1682 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:47.614372 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607839d-ae05-4a14-aac7-d7711957909b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:47.615511 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607839d-ae05-4a14-aac7-d7711957909b service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:47.615936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607839d-ae05-4a14-aac7-d7711957909b service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:47.616421 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9607839d-ae05-4a14-aac7-d7711957909b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:47.637339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9607839d-ae05-4a14-aac7-d7711957909b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:47.638852 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9607839d-ae05-4a14-aac7-d7711957909b service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:47.650850 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1152/2313] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:05:48.428578 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e235054-090a-4a19-ac54-1418d1cbca2a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:48.431006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8e235054-090a-4a19-ac54-1418d1cbca2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:48.465705 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e235054-090a-4a19-ac54-1418d1cbca2a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:48.494824 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e38d15-3300-4db5-b044-de3c6b92665b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:48.496503 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e38d15-3300-4db5-b044-de3c6b92665b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:48.497327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e38d15-3300-4db5-b044-de3c6b92665b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:48.503947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e38d15-3300-4db5-b044-de3c6b92665b admin admin] PATH_INFO: `/v3/projects/a0495e2abcab48aabdf623183f87717e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:48.506740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e38d15-3300-4db5-b044-de3c6b92665b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0495e2abcab48aabdf623183f87717e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:48.522608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e38d15-3300-4db5-b044-de3c6b92665b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:48.804492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-22e38d15-3300-4db5-b044-de3c6b92665b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0495e2abcab48aabdf623183f87717e', 'request_id': 'req-22e38d15-3300-4db5-b044-de3c6b92665b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:48.816858 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-22e38d15-3300-4db5-b044-de3c6b92665b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0495e2abcab48aabdf623183f87717e', 'request_id': 'req-22e38d15-3300-4db5-b044-de3c6b92665b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:49.071379 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-22e38d15-3300-4db5-b044-de3c6b92665b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0495e2abcab48aabdf623183f87717e', 'request_id': 'req-22e38d15-3300-4db5-b044-de3c6b92665b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:49.113127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-22e38d15-3300-4db5-b044-de3c6b92665b admin admin] The token cache is being invalidate because project a0495e2abcab48aabdf623183f87717e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:49.114033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-22e38d15-3300-4db5-b044-de3c6b92665b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:49.115434 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-22e38d15-3300-4db5-b044-de3c6b92665b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:49.122881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1162/2314] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:05:48 2022] DELETE /identity/v3/projects/a0495e2abcab48aabdf623183f87717e => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:49.123982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607839d-ae05-4a14-aac7-d7711957909b service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:49.143250 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9607839d-ae05-4a14-aac7-d7711957909b service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:49.263693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87209e5c-e480-4d90-9972-ff51a965746b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:49.263693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87209e5c-e480-4d90-9972-ff51a965746b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:49.263693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87209e5c-e480-4d90-9972-ff51a965746b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:49.263693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87209e5c-e480-4d90-9972-ff51a965746b admin admin] PATH_INFO: `/v3/users/99529005814945cbb4a0030175743b36` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:49.264431 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87209e5c-e480-4d90-9972-ff51a965746b admin admin] RBAC: Authorizing `identity:delete_user(user_id=99529005814945cbb4a0030175743b36)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:49.264431 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87209e5c-e480-4d90-9972-ff51a965746b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:49.519635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:49.683291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-87209e5c-e480-4d90-9972-ff51a965746b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99529005814945cbb4a0030175743b36'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:49.864366 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-87209e5c-e480-4d90-9972-ff51a965746b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99529005814945cbb4a0030175743b36'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:49.865254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-87209e5c-e480-4d90-9972-ff51a965746b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99529005814945cbb4a0030175743b36'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:49.865930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-87209e5c-e480-4d90-9972-ff51a965746b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99529005814945cbb4a0030175743b36'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:49.916641 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-87209e5c-e480-4d90-9972-ff51a965746b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99529005814945cbb4a0030175743b36'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:49.974263 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1153/2315] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:05:49 2022] DELETE /identity/v3/users/99529005814945cbb4a0030175743b36 => generated 0 bytes in 850 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:50.492121 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:50.717844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e38d15-3300-4db5-b044-de3c6b92665b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:50.726712 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-22e38d15-3300-4db5-b044-de3c6b92665b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:50.747144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e38d15-3300-4db5-b044-de3c6b92665b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:50.814800 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5651a572-8709-4ce9-9ac5-0c536a1d26fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:50.815456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651a572-8709-4ce9-9ac5-0c536a1d26fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:50.815781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651a572-8709-4ce9-9ac5-0c536a1d26fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:50.816104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5651a572-8709-4ce9-9ac5-0c536a1d26fe admin admin] PATH_INFO: `/v3/projects/e2ceb78d4f2843879f55a6c9093b78f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:50.817275 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651a572-8709-4ce9-9ac5-0c536a1d26fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2ceb78d4f2843879f55a6c9093b78f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:50.819356 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5651a572-8709-4ce9-9ac5-0c536a1d26fe admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:50.979997 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5651a572-8709-4ce9-9ac5-0c536a1d26fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2ceb78d4f2843879f55a6c9093b78f7', 'request_id': 'req-5651a572-8709-4ce9-9ac5-0c536a1d26fe'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:50.988838 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5651a572-8709-4ce9-9ac5-0c536a1d26fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2ceb78d4f2843879f55a6c9093b78f7', 'request_id': 'req-5651a572-8709-4ce9-9ac5-0c536a1d26fe'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:51.149160 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5651a572-8709-4ce9-9ac5-0c536a1d26fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2ceb78d4f2843879f55a6c9093b78f7', 'request_id': 'req-5651a572-8709-4ce9-9ac5-0c536a1d26fe'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:51.166359 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5651a572-8709-4ce9-9ac5-0c536a1d26fe admin admin] The token cache is being invalidate because project e2ceb78d4f2843879f55a6c9093b78f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:05:51.166651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5651a572-8709-4ce9-9ac5-0c536a1d26fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:51.167802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5651a572-8709-4ce9-9ac5-0c536a1d26fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:05:51.169365 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1163/2316] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:05:50 2022] DELETE /identity/v3/projects/e2ceb78d4f2843879f55a6c9093b78f7 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:51.174877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843ea7f-a2a7-4e1a-bde2-5e95e78fd928 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:51.175552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843ea7f-a2a7-4e1a-bde2-5e95e78fd928 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:51.175552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0843ea7f-a2a7-4e1a-bde2-5e95e78fd928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:51.205868 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0843ea7f-a2a7-4e1a-bde2-5e95e78fd928 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:51.207131 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0843ea7f-a2a7-4e1a-bde2-5e95e78fd928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:51.285420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1154/2317] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:51.299902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f8e1d-dcc2-4d81-b210-97d09de39227 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:51.300632 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f8e1d-dcc2-4d81-b210-97d09de39227 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:51.300911 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f8e1d-dcc2-4d81-b210-97d09de39227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:51.321896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5d6f8e1d-dcc2-4d81-b210-97d09de39227 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:51.323143 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5d6f8e1d-dcc2-4d81-b210-97d09de39227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:51.332240 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1164/2318] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:51.337729 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843ea7f-a2a7-4e1a-bde2-5e95e78fd928 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:51.348465 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0843ea7f-a2a7-4e1a-bde2-5e95e78fd928 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:51.353229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee1d82-a7b7-4761-b895-eae3964c8e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:51.353840 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee1d82-a7b7-4761-b895-eae3964c8e8f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:51.354192 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee1d82-a7b7-4761-b895-eae3964c8e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:51.354470 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee1d82-a7b7-4761-b895-eae3964c8e8f admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:51.355716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee1d82-a7b7-4761-b895-eae3964c8e8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:05:51.355716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee1d82-a7b7-4761-b895-eae3964c8e8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:51.357093 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee1d82-a7b7-4761-b895-eae3964c8e8f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:51.371463 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1155/2319] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:05:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:05:51.376918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f8e1d-dcc2-4d81-b210-97d09de39227 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:51.389927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f8e1d-dcc2-4d81-b210-97d09de39227 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:51.395517 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d94079-1b65-49d7-837a-0480b682c48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:51.396173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d94079-1b65-49d7-837a-0480b682c48d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:51.396463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d94079-1b65-49d7-837a-0480b682c48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:51.396736 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d94079-1b65-49d7-837a-0480b682c48d admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:51.397380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d94079-1b65-49d7-837a-0480b682c48d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:51.399988 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d94079-1b65-49d7-837a-0480b682c48d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:51.564998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1165/2320] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:05:51 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:51.574749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee1d82-a7b7-4761-b895-eae3964c8e8f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:51.586125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee1d82-a7b7-4761-b895-eae3964c8e8f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:51.653550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751ae67-4321-447e-a6fa-3e421c9db71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:51.654114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751ae67-4321-447e-a6fa-3e421c9db71e admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:51.654563 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751ae67-4321-447e-a6fa-3e421c9db71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:51.654872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751ae67-4321-447e-a6fa-3e421c9db71e admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:51.655608 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751ae67-4321-447e-a6fa-3e421c9db71e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:51.657566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751ae67-4321-447e-a6fa-3e421c9db71e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:51.942002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1156/2321] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:05:51 2022] POST /identity/v3/users => generated 294 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:51.942745 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d94079-1b65-49d7-837a-0480b682c48d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:51.949777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d94079-1b65-49d7-837a-0480b682c48d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:51.956139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bcd35-e07e-44a2-96f8-8eb02f94855a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:51.956773 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bcd35-e07e-44a2-96f8-8eb02f94855a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:51.957051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bcd35-e07e-44a2-96f8-8eb02f94855a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:51.957344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1bcd35-e07e-44a2-96f8-8eb02f94855a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:51.957975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1bcd35-e07e-44a2-96f8-8eb02f94855a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:51.960077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1bcd35-e07e-44a2-96f8-8eb02f94855a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:51.979385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1166/2322] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:51.980675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751ae67-4321-447e-a6fa-3e421c9db71e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:52.011187 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751ae67-4321-447e-a6fa-3e421c9db71e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:52.018212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f69fa0-27a0-4d92-ba81-8dfe286f689f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:52.019529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f69fa0-27a0-4d92-ba81-8dfe286f689f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:52.020063 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f69fa0-27a0-4d92-ba81-8dfe286f689f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:52.027446 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f69fa0-27a0-4d92-ba81-8dfe286f689f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:52.027845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f69fa0-27a0-4d92-ba81-8dfe286f689f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:52.029933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f69fa0-27a0-4d92-ba81-8dfe286f689f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:52.048212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1157/2323] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:52.055965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bcd35-e07e-44a2-96f8-8eb02f94855a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:52.068531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1bcd35-e07e-44a2-96f8-8eb02f94855a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:52.073803 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8f155-574a-47ce-8baa-bb5bf9ebcb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:52.074596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8f155-574a-47ce-8baa-bb5bf9ebcb81 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:52.074880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8f155-574a-47ce-8baa-bb5bf9ebcb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:52.075140 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd8f155-574a-47ce-8baa-bb5bf9ebcb81 admin admin] PATH_INFO: `/v3/projects/9a38597d4f364326b91828b5786cfe49/users/5a91b3b4d37d4039b85e655132a12e08/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:52.098454 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8f155-574a-47ce-8baa-bb5bf9ebcb81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a38597d4f364326b91828b5786cfe49, user_id=5a91b3b4d37d4039b85e655132a12e08, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:52.103613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd8f155-574a-47ce-8baa-bb5bf9ebcb81 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:52.304263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1167/2324] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:52 2022] PUT /identity/v3/projects/9a38597d4f364326b91828b5786cfe49/users/5a91b3b4d37d4039b85e655132a12e08/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:52.315335 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7785be-f905-4046-967d-4c5193719861 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:52.315968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7785be-f905-4046-967d-4c5193719861 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:52.315968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7785be-f905-4046-967d-4c5193719861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:52.353766 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3e7785be-f905-4046-967d-4c5193719861 None None] MFA Rules not processed for user `5a91b3b4d37d4039b85e655132a12e08`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:52.355051 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3e7785be-f905-4046-967d-4c5193719861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:52.454724 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1158/2325] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:52.457491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8f155-574a-47ce-8baa-bb5bf9ebcb81 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:52.533426 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd8f155-574a-47ce-8baa-bb5bf9ebcb81 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:52.538422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebcb1f-9573-4067-b452-39950fcc2ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:52.539454 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebcb1f-9573-4067-b452-39950fcc2ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:52.539747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebcb1f-9573-4067-b452-39950fcc2ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:52.540024 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebcb1f-9573-4067-b452-39950fcc2ae6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:52.540668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebcb1f-9573-4067-b452-39950fcc2ae6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:52.543046 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebcb1f-9573-4067-b452-39950fcc2ae6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:52.670880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1168/2326] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:05:52 2022] POST /identity/v3/projects => generated 387 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:05:52.677181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7785be-f905-4046-967d-4c5193719861 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:52.693818 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7785be-f905-4046-967d-4c5193719861 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:52.757527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9a067-955c-476f-9b3b-0bd7a2a1073b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:52.758139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9a067-955c-476f-9b3b-0bd7a2a1073b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:52.758482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9a067-955c-476f-9b3b-0bd7a2a1073b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:52.758838 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9a067-955c-476f-9b3b-0bd7a2a1073b admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:52.759698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9a067-955c-476f-9b3b-0bd7a2a1073b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:52.761663 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9a067-955c-476f-9b3b-0bd7a2a1073b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:52.854905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1159/2327] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:05:52 2022] POST /identity/v3/users => generated 294 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:52.859004 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebcb1f-9573-4067-b452-39950fcc2ae6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:52.870402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebcb1f-9573-4067-b452-39950fcc2ae6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:52.876373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd7063-6ac5-46f7-8e74-2912829cc7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:52.877187 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd7063-6ac5-46f7-8e74-2912829cc7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:52.877469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd7063-6ac5-46f7-8e74-2912829cc7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:52.877746 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdd7063-6ac5-46f7-8e74-2912829cc7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:52.878395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd7063-6ac5-46f7-8e74-2912829cc7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:52.880084 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdd7063-6ac5-46f7-8e74-2912829cc7a9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:52.892579 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1169/2328] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:52.898239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9a067-955c-476f-9b3b-0bd7a2a1073b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:52.909564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9a067-955c-476f-9b3b-0bd7a2a1073b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:52.914653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17ad18-d733-43fd-ae7e-48759cd7678e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:52.915264 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17ad18-d733-43fd-ae7e-48759cd7678e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:52.915546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17ad18-d733-43fd-ae7e-48759cd7678e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:52.915858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17ad18-d733-43fd-ae7e-48759cd7678e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:52.916770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17ad18-d733-43fd-ae7e-48759cd7678e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:52.918388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17ad18-d733-43fd-ae7e-48759cd7678e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:52.930302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1160/2329] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:52.934527 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd7063-6ac5-46f7-8e74-2912829cc7a9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:52.945617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdd7063-6ac5-46f7-8e74-2912829cc7a9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:52.950347 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d78b4-9a52-4ff6-8dc2-91ce233289fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:52.951043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d78b4-9a52-4ff6-8dc2-91ce233289fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:52.951313 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d78b4-9a52-4ff6-8dc2-91ce233289fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:52.951587 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d78b4-9a52-4ff6-8dc2-91ce233289fe admin admin] PATH_INFO: `/v3/projects/89d41f3248a84f169b8240fdfa62ef92/users/a2a45c7ee8234cfba1dd273fb5682177/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:52.972555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d78b4-9a52-4ff6-8dc2-91ce233289fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=89d41f3248a84f169b8240fdfa62ef92, user_id=a2a45c7ee8234cfba1dd273fb5682177, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:52.978377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d78b4-9a52-4ff6-8dc2-91ce233289fe admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:53.115036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1170/2330] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:52 2022] PUT /identity/v3/projects/89d41f3248a84f169b8240fdfa62ef92/users/a2a45c7ee8234cfba1dd273fb5682177/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:53.137103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fee9d-7ab4-4810-ab82-82948460b2bd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:53.137658 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fee9d-7ab4-4810-ab82-82948460b2bd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:53.137975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946fee9d-7ab4-4810-ab82-82948460b2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:53.178737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-946fee9d-7ab4-4810-ab82-82948460b2bd None None] MFA Rules not processed for user `a2a45c7ee8234cfba1dd273fb5682177`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:53.180002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-946fee9d-7ab4-4810-ab82-82948460b2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:53.274158 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1161/2331] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:53.281005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d78b4-9a52-4ff6-8dc2-91ce233289fe admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:53.338312 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:53.379139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d78b4-9a52-4ff6-8dc2-91ce233289fe admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:53.384539 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19fe9e-8e84-4712-ab9d-177e8d101a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:53.385287 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19fe9e-8e84-4712-ab9d-177e8d101a30 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:53.385730 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19fe9e-8e84-4712-ab9d-177e8d101a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:53.386118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed19fe9e-8e84-4712-ab9d-177e8d101a30 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:53.386952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19fe9e-8e84-4712-ab9d-177e8d101a30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:53.389199 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed19fe9e-8e84-4712-ab9d-177e8d101a30 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:53.525006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1171/2332] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:05:53 2022] POST /identity/v3/projects => generated 387 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:53.526396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fee9d-7ab4-4810-ab82-82948460b2bd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:53.538759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946fee9d-7ab4-4810-ab82-82948460b2bd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:53.647222 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c90c9b-ede2-4958-90bf-dd192f2cbadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:53.651895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c90c9b-ede2-4958-90bf-dd192f2cbadd admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:53.651895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c90c9b-ede2-4958-90bf-dd192f2cbadd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:53.651895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c90c9b-ede2-4958-90bf-dd192f2cbadd admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:53.651895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c90c9b-ede2-4958-90bf-dd192f2cbadd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:53.651895 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c90c9b-ede2-4958-90bf-dd192f2cbadd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:53.921560 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1162/2333] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:05:53 2022] POST /identity/v3/users => generated 300 bytes in 392 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:05:53.923933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19fe9e-8e84-4712-ab9d-177e8d101a30 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:53.939611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed19fe9e-8e84-4712-ab9d-177e8d101a30 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:53.944639 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb23cd4-d336-4a99-875c-58f7f4fc2857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:53.945292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb23cd4-d336-4a99-875c-58f7f4fc2857 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:53.945563 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb23cd4-d336-4a99-875c-58f7f4fc2857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:53.946285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb23cd4-d336-4a99-875c-58f7f4fc2857 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:53.946604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb23cd4-d336-4a99-875c-58f7f4fc2857 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:53.948134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb23cd4-d336-4a99-875c-58f7f4fc2857 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:53.961552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1172/2334] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:53.970978 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c90c9b-ede2-4958-90bf-dd192f2cbadd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:53.986350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c90c9b-ede2-4958-90bf-dd192f2cbadd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:53.994355 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6500318-0a7c-4198-9e8d-c06d4ed2de7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:53.995308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6500318-0a7c-4198-9e8d-c06d4ed2de7b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:53.995714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6500318-0a7c-4198-9e8d-c06d4ed2de7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:53.996078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6500318-0a7c-4198-9e8d-c06d4ed2de7b admin admin] PATH_INFO: `/v3/projects/b44098cd716440028d9795ed785be3bb/users/ef3744018d1845a08a540f40db6b08b5/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:54.015026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6500318-0a7c-4198-9e8d-c06d4ed2de7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44098cd716440028d9795ed785be3bb, user_id=ef3744018d1845a08a540f40db6b08b5, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:54.018433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6500318-0a7c-4198-9e8d-c06d4ed2de7b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:54.118415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1163/2335] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:53 2022] PUT /identity/v3/projects/b44098cd716440028d9795ed785be3bb/users/ef3744018d1845a08a540f40db6b08b5/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:05:54.123147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb23cd4-d336-4a99-875c-58f7f4fc2857 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:54.137666 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb23cd4-d336-4a99-875c-58f7f4fc2857 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:54.255556 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e78cf2-f85c-445d-ba2c-cebd2ecf6290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:54.256220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e78cf2-f85c-445d-ba2c-cebd2ecf6290 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:54.256507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e78cf2-f85c-445d-ba2c-cebd2ecf6290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:54.256785 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e78cf2-f85c-445d-ba2c-cebd2ecf6290 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:54.257481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e78cf2-f85c-445d-ba2c-cebd2ecf6290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:54.259436 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e78cf2-f85c-445d-ba2c-cebd2ecf6290 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:54.275040 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1173/2336] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:05:54.299015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6500318-0a7c-4198-9e8d-c06d4ed2de7b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:54.299015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6500318-0a7c-4198-9e8d-c06d4ed2de7b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:54.299015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85237c12-3f9b-44e2-b5d3-8d6d5cb6e9a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:54.299015 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85237c12-3f9b-44e2-b5d3-8d6d5cb6e9a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:54.299638 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85237c12-3f9b-44e2-b5d3-8d6d5cb6e9a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:54.299638 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85237c12-3f9b-44e2-b5d3-8d6d5cb6e9a2 admin admin] PATH_INFO: `/v3/projects/b44098cd716440028d9795ed785be3bb/users/ef3744018d1845a08a540f40db6b08b5/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:54.301696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85237c12-3f9b-44e2-b5d3-8d6d5cb6e9a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44098cd716440028d9795ed785be3bb, user_id=ef3744018d1845a08a540f40db6b08b5, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:54.304871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85237c12-3f9b-44e2-b5d3-8d6d5cb6e9a2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:54.328979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1164/2337] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:05:54 2022] PUT /identity/v3/projects/b44098cd716440028d9795ed785be3bb/users/ef3744018d1845a08a540f40db6b08b5/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:05:54.338466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa02346-689b-4b4c-bfcc-a010aee2109e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:54.338760 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa02346-689b-4b4c-bfcc-a010aee2109e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:54.339039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa02346-689b-4b4c-bfcc-a010aee2109e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:54.389815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9aa02346-689b-4b4c-bfcc-a010aee2109e None None] MFA Rules not processed for user `ef3744018d1845a08a540f40db6b08b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:54.391519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9aa02346-689b-4b4c-bfcc-a010aee2109e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:54.524265 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1174/2338] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:54 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:05:54.538128 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85237c12-3f9b-44e2-b5d3-8d6d5cb6e9a2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:54.540343 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-85237c12-3f9b-44e2-b5d3-8d6d5cb6e9a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:54.614292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:54.649856 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85237c12-3f9b-44e2-b5d3-8d6d5cb6e9a2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:54.654675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a937e6-1594-4dab-9781-0fc0f2f18ba2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:54.655433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a937e6-1594-4dab-9781-0fc0f2f18ba2 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:54.655908 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a937e6-1594-4dab-9781-0fc0f2f18ba2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:54.656258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a937e6-1594-4dab-9781-0fc0f2f18ba2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:54.668271 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a937e6-1594-4dab-9781-0fc0f2f18ba2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:54.670090 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a937e6-1594-4dab-9781-0fc0f2f18ba2 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:54.682465 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1165/2339] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:54 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:55.667269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497b995-1f3c-4f48-9c26-5349b71bd622 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:55.667849 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497b995-1f3c-4f48-9c26-5349b71bd622 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:55.667849 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4497b995-1f3c-4f48-9c26-5349b71bd622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:55.691136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4497b995-1f3c-4f48-9c26-5349b71bd622 None None] MFA Rules not processed for user `349bc0a30f324816804e596c937e3da0`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:05:55.693400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4497b995-1f3c-4f48-9c26-5349b71bd622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:05:55.775157 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1175/2340] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:05:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:05:57.321793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a937e6-1594-4dab-9781-0fc0f2f18ba2 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:05:57.342845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a937e6-1594-4dab-9781-0fc0f2f18ba2 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:05:57.349325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3481e0-a920-48f4-b4b7-3730cc88cd07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:05:57.350277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3481e0-a920-48f4-b4b7-3730cc88cd07 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:05:57.350698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3481e0-a920-48f4-b4b7-3730cc88cd07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:05:57.351046 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3481e0-a920-48f4-b4b7-3730cc88cd07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:05:57.481407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3481e0-a920-48f4-b4b7-3730cc88cd07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:05:57.483711 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3481e0-a920-48f4-b4b7-3730cc88cd07 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:05:57.497099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1166/2341] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:05:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:05:58.404676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:05:59.054080 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:00.036468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497b995-1f3c-4f48-9c26-5349b71bd622 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:00.037938 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4497b995-1f3c-4f48-9c26-5349b71bd622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:00.314912 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4497b995-1f3c-4f48-9c26-5349b71bd622 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:00.332032 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba028949-bcc3-48d6-a5e6-26eb9e7f89dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:00.337158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba028949-bcc3-48d6-a5e6-26eb9e7f89dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:00.337158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba028949-bcc3-48d6-a5e6-26eb9e7f89dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:00.337158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba028949-bcc3-48d6-a5e6-26eb9e7f89dd admin admin] PATH_INFO: `/v3/users/b74b92ce24d34b7e9db2c8394d458909` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:00.337158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba028949-bcc3-48d6-a5e6-26eb9e7f89dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=b74b92ce24d34b7e9db2c8394d458909)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:00.343615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba028949-bcc3-48d6-a5e6-26eb9e7f89dd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:00.814635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ba028949-bcc3-48d6-a5e6-26eb9e7f89dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b74b92ce24d34b7e9db2c8394d458909'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:00.993977 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ba028949-bcc3-48d6-a5e6-26eb9e7f89dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b74b92ce24d34b7e9db2c8394d458909'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:00.995617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ba028949-bcc3-48d6-a5e6-26eb9e7f89dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b74b92ce24d34b7e9db2c8394d458909'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:00.997271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ba028949-bcc3-48d6-a5e6-26eb9e7f89dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b74b92ce24d34b7e9db2c8394d458909'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:01.043055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ba028949-bcc3-48d6-a5e6-26eb9e7f89dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b74b92ce24d34b7e9db2c8394d458909'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:01.071454 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1176/2342] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:00 2022] DELETE /identity/v3/users/b74b92ce24d34b7e9db2c8394d458909 => generated 0 bytes in 1035 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:01.077639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6340926-b263-4b87-aa0d-8ae7cf8c2c32 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:01.083983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6340926-b263-4b87-aa0d-8ae7cf8c2c32 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:01.083983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6340926-b263-4b87-aa0d-8ae7cf8c2c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:01.112228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d6340926-b263-4b87-aa0d-8ae7cf8c2c32 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:01.114478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d6340926-b263-4b87-aa0d-8ae7cf8c2c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:01.231845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1167/2343] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:01.247529 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba028949-bcc3-48d6-a5e6-26eb9e7f89dd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:01.249563 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ba028949-bcc3-48d6-a5e6-26eb9e7f89dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:01.255234 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de82069-379b-4bea-a290-99fbfded8ed4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:01.255710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de82069-379b-4bea-a290-99fbfded8ed4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:01.256169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de82069-379b-4bea-a290-99fbfded8ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:01.290225 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2de82069-379b-4bea-a290-99fbfded8ed4 None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:01.292763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2de82069-379b-4bea-a290-99fbfded8ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:01.381130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba028949-bcc3-48d6-a5e6-26eb9e7f89dd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:01.394485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d15383-6fc8-47b4-b533-f3f646c478a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:01.395671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d15383-6fc8-47b4-b533-f3f646c478a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:01.396153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d15383-6fc8-47b4-b533-f3f646c478a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:01.396872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d15383-6fc8-47b4-b533-f3f646c478a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:01.407354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1168/2344] 10.222.0.67 () {58 vars in 1009 bytes} [Mon Sep 26 18:06:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:01.417996 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d15383-6fc8-47b4-b533-f3f646c478a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:01.425080 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d15383-6fc8-47b4-b533-f3f646c478a2 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:01.441936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1177/2345] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:01.524495 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de82069-379b-4bea-a290-99fbfded8ed4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:01.528307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2de82069-379b-4bea-a290-99fbfded8ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:01.716093 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de82069-379b-4bea-a290-99fbfded8ed4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:01.727221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe23ea07-eb71-430a-a1ec-0d5aaa350632 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:01.731325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23ea07-eb71-430a-a1ec-0d5aaa350632 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:01.731325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23ea07-eb71-430a-a1ec-0d5aaa350632 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:01.731325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe23ea07-eb71-430a-a1ec-0d5aaa350632 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:01.756818 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23ea07-eb71-430a-a1ec-0d5aaa350632 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:01.758934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe23ea07-eb71-430a-a1ec-0d5aaa350632 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:01.780823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1169/2346] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:06:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:06:02.554643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d15383-6fc8-47b4-b533-f3f646c478a2 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:02.556097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b9d15383-6fc8-47b4-b533-f3f646c478a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:02.594417 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d15383-6fc8-47b4-b533-f3f646c478a2 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:02.615030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29793c1d-69c1-4f5a-b40e-9c5ef357294e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:02.615725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29793c1d-69c1-4f5a-b40e-9c5ef357294e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:02.616327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29793c1d-69c1-4f5a-b40e-9c5ef357294e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:02.616814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29793c1d-69c1-4f5a-b40e-9c5ef357294e admin admin] PATH_INFO: `/v3/projects/71ab740a9b3b458e9bb33bbbe4ad24f4` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:02.619155 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29793c1d-69c1-4f5a-b40e-9c5ef357294e admin admin] RBAC: Authorizing `identity:delete_project(project_id=71ab740a9b3b458e9bb33bbbe4ad24f4)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:02.621396 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29793c1d-69c1-4f5a-b40e-9c5ef357294e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:02.829203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29793c1d-69c1-4f5a-b40e-9c5ef357294e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71ab740a9b3b458e9bb33bbbe4ad24f4', 'request_id': 'req-29793c1d-69c1-4f5a-b40e-9c5ef357294e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:02.832038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac33e1b-a2b8-4a02-ae0f-e4642ff1621c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:02.832780 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac33e1b-a2b8-4a02-ae0f-e4642ff1621c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:02.834426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac33e1b-a2b8-4a02-ae0f-e4642ff1621c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:02.846853 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29793c1d-69c1-4f5a-b40e-9c5ef357294e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71ab740a9b3b458e9bb33bbbe4ad24f4', 'request_id': 'req-29793c1d-69c1-4f5a-b40e-9c5ef357294e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:02.898776 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9ac33e1b-a2b8-4a02-ae0f-e4642ff1621c None None] MFA Rules not processed for user `97ed136f68c540abbdaf04cbe34a682e`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:02.900346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9ac33e1b-a2b8-4a02-ae0f-e4642ff1621c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:03.001984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29793c1d-69c1-4f5a-b40e-9c5ef357294e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71ab740a9b3b458e9bb33bbbe4ad24f4', 'request_id': 'req-29793c1d-69c1-4f5a-b40e-9c5ef357294e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:03.003037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:03.028624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29793c1d-69c1-4f5a-b40e-9c5ef357294e admin admin] The token cache is being invalidate because project 71ab740a9b3b458e9bb33bbbe4ad24f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:03.029204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29793c1d-69c1-4f5a-b40e-9c5ef357294e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:03.030186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29793c1d-69c1-4f5a-b40e-9c5ef357294e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:03.032951 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1178/2347] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:02 2022] DELETE /identity/v3/projects/71ab740a9b3b458e9bb33bbbe4ad24f4 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:03.044213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb33d3c-eaef-47c8-a2a9-96081b7b4b52 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:03.044645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb33d3c-eaef-47c8-a2a9-96081b7b4b52 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:03.045112 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb33d3c-eaef-47c8-a2a9-96081b7b4b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:03.074296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7fb33d3c-eaef-47c8-a2a9-96081b7b4b52 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:03.074828 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1170/2348] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:03.075569 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7fb33d3c-eaef-47c8-a2a9-96081b7b4b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:03.236163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1179/2349] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:03.247187 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3b7f0-0988-44b2-b432-d7601e466547 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:03.247697 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3b7f0-0988-44b2-b432-d7601e466547 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:03.248636 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c3b7f0-0988-44b2-b432-d7601e466547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:03.310578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-87c3b7f0-0988-44b2-b432-d7601e466547 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:03.314062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-87c3b7f0-0988-44b2-b432-d7601e466547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:03.337934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1171/2350] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:03.342266 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb33d3c-eaef-47c8-a2a9-96081b7b4b52 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:03.357073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb33d3c-eaef-47c8-a2a9-96081b7b4b52 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:03.365175 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09b8ef2-72ba-461a-afa5-b13c86da0e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:03.366242 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b8ef2-72ba-461a-afa5-b13c86da0e81 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:03.366728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b8ef2-72ba-461a-afa5-b13c86da0e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:03.367033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09b8ef2-72ba-461a-afa5-b13c86da0e81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:03.367778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b8ef2-72ba-461a-afa5-b13c86da0e81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:06:03.368055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b8ef2-72ba-461a-afa5-b13c86da0e81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:03.369742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09b8ef2-72ba-461a-afa5-b13c86da0e81 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:03.384346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1180/2351] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:06:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:06:03.394778 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3b7f0-0988-44b2-b432-d7601e466547 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:03.412075 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c3b7f0-0988-44b2-b432-d7601e466547 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:03.424841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efd5c5-ddf2-415d-8667-afa76d05e0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:03.427842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efd5c5-ddf2-415d-8667-afa76d05e0c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:03.429213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efd5c5-ddf2-415d-8667-afa76d05e0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:03.429738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efd5c5-ddf2-415d-8667-afa76d05e0c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:03.430657 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efd5c5-ddf2-415d-8667-afa76d05e0c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:03.434090 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25efd5c5-ddf2-415d-8667-afa76d05e0c8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:03.576658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca253019-b79a-4964-81e5-f115dec04815 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:03.577956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca253019-b79a-4964-81e5-f115dec04815 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:03.579070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca253019-b79a-4964-81e5-f115dec04815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:03.644268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ca253019-b79a-4964-81e5-f115dec04815 None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:03.644268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ca253019-b79a-4964-81e5-f115dec04815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:03.730276 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1181/2352] 10.222.0.67 () {58 vars in 967 bytes} [Mon Sep 26 18:06:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:03.764075 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca253019-b79a-4964-81e5-f115dec04815 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:03.768353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1172/2353] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:03 2022] POST /identity/v3/projects => generated 371 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:03.775234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ca253019-b79a-4964-81e5-f115dec04815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:03.776840 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efd5c5-ddf2-415d-8667-afa76d05e0c8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:03.921747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efd5c5-ddf2-415d-8667-afa76d05e0c8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:03.928459 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850fe2b-b703-4546-a18f-5e6865731797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:03.940568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850fe2b-b703-4546-a18f-5e6865731797 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:03.941825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850fe2b-b703-4546-a18f-5e6865731797 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:03.945269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850fe2b-b703-4546-a18f-5e6865731797 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:03.946507 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850fe2b-b703-4546-a18f-5e6865731797 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:03.948879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca253019-b79a-4964-81e5-f115dec04815 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:03.949705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850fe2b-b703-4546-a18f-5e6865731797 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:04.081395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e557fde-a34d-4a76-850f-e6351d882edb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:04.082306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e557fde-a34d-4a76-850f-e6351d882edb service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:04.082730 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e557fde-a34d-4a76-850f-e6351d882edb service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:04.083082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e557fde-a34d-4a76-850f-e6351d882edb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:04.186995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1173/2354] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:03 2022] POST /identity/v3/users => generated 286 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:04.193367 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850fe2b-b703-4546-a18f-5e6865731797 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:04.232478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850fe2b-b703-4546-a18f-5e6865731797 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:04.254703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822cfbe-ddd2-4ecf-8d58-ad2fae51c503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:04.257852 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822cfbe-ddd2-4ecf-8d58-ad2fae51c503 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:04.259523 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822cfbe-ddd2-4ecf-8d58-ad2fae51c503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:04.260696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9822cfbe-ddd2-4ecf-8d58-ad2fae51c503 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:04.269776 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822cfbe-ddd2-4ecf-8d58-ad2fae51c503 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:04.272153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9822cfbe-ddd2-4ecf-8d58-ad2fae51c503 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:04.277806 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e557fde-a34d-4a76-850f-e6351d882edb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:04.281150 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e557fde-a34d-4a76-850f-e6351d882edb service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:04.291496 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1174/2355] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:04.296383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822cfbe-ddd2-4ecf-8d58-ad2fae51c503 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:04.298136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1182/2356] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:06:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:06:04.310981 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9822cfbe-ddd2-4ecf-8d58-ad2fae51c503 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:04.319827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26022190-cd93-43cd-97ee-2b851e25e3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:04.320669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26022190-cd93-43cd-97ee-2b851e25e3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:04.321330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26022190-cd93-43cd-97ee-2b851e25e3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:04.321844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26022190-cd93-43cd-97ee-2b851e25e3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:04.323042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26022190-cd93-43cd-97ee-2b851e25e3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:04.324715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:04.325430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26022190-cd93-43cd-97ee-2b851e25e3c1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:04.356205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1175/2357] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:04 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:06:04.361503 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e557fde-a34d-4a76-850f-e6351d882edb service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:04.385631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e557fde-a34d-4a76-850f-e6351d882edb service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:04.394660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a1cdb-9cdd-472e-968b-892238699f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:04.396005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a1cdb-9cdd-472e-968b-892238699f60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:04.396506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a1cdb-9cdd-472e-968b-892238699f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:04.396998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a1cdb-9cdd-472e-968b-892238699f60 admin admin] PATH_INFO: `/v3/projects/1e9d16b034454af791f2d947bc22aff6/users/11e33b29101f4bcbb4bd204679f46d68/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:04.428717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a1cdb-9cdd-472e-968b-892238699f60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e9d16b034454af791f2d947bc22aff6, user_id=11e33b29101f4bcbb4bd204679f46d68, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:04.433343 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a1cdb-9cdd-472e-968b-892238699f60 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:04.526097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26022190-cd93-43cd-97ee-2b851e25e3c1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:04.528557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-26022190-cd93-43cd-97ee-2b851e25e3c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:04.634284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1183/2358] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:04 2022] PUT /identity/v3/projects/1e9d16b034454af791f2d947bc22aff6/users/11e33b29101f4bcbb4bd204679f46d68/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:04.651427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3275a3-4cb3-4f18-92d0-3fa7f03af88f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:04.652474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3275a3-4cb3-4f18-92d0-3fa7f03af88f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:04.653152 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3275a3-4cb3-4f18-92d0-3fa7f03af88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:04.665402 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26022190-cd93-43cd-97ee-2b851e25e3c1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:04.707581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0e3275a3-4cb3-4f18-92d0-3fa7f03af88f None None] MFA Rules not processed for user `11e33b29101f4bcbb4bd204679f46d68`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:04.709062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0e3275a3-4cb3-4f18-92d0-3fa7f03af88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:04.778849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7d44b-1f17-4beb-8d81-7416e55afab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:04.780631 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7d44b-1f17-4beb-8d81-7416e55afab4 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:04.784215 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7d44b-1f17-4beb-8d81-7416e55afab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:04.784932 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7d44b-1f17-4beb-8d81-7416e55afab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:04.795924 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cba7d44b-1f17-4beb-8d81-7416e55afab4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:04.858005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1184/2359] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:04 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:04.860691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3275a3-4cb3-4f18-92d0-3fa7f03af88f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:04.938172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7d44b-1f17-4beb-8d81-7416e55afab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:04.942315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7d44b-1f17-4beb-8d81-7416e55afab4 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:04.956159 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1176/2360] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:04 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:06:04.977269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3275a3-4cb3-4f18-92d0-3fa7f03af88f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:04.990141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab903ac-5d18-489e-86af-5c8e7f6a7c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:04.991385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab903ac-5d18-489e-86af-5c8e7f6a7c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:04.992195 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab903ac-5d18-489e-86af-5c8e7f6a7c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:04.992603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab903ac-5d18-489e-86af-5c8e7f6a7c01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:04.994662 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab903ac-5d18-489e-86af-5c8e7f6a7c01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:05.004915 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab903ac-5d18-489e-86af-5c8e7f6a7c01 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:05.185304 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1185/2361] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:04 2022] POST /identity/v3/projects => generated 373 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:05.189494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7d44b-1f17-4beb-8d81-7416e55afab4 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:05.210221 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7d44b-1f17-4beb-8d81-7416e55afab4 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:05.372431 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a8943-e2f2-40bc-b1a1-3739b5d6ae31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:05.373874 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a8943-e2f2-40bc-b1a1-3739b5d6ae31 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:05.374513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a8943-e2f2-40bc-b1a1-3739b5d6ae31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:05.382173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5a8943-e2f2-40bc-b1a1-3739b5d6ae31 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:05.382173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a8943-e2f2-40bc-b1a1-3739b5d6ae31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:05.382173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5a8943-e2f2-40bc-b1a1-3739b5d6ae31 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:05.539711 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1177/2362] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:05 2022] POST /identity/v3/users => generated 293 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:05.541546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab903ac-5d18-489e-86af-5c8e7f6a7c01 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:05.554026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab903ac-5d18-489e-86af-5c8e7f6a7c01 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:05.560406 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d682de0-e130-4b1f-b71d-98bbab10a8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:05.571194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d682de0-e130-4b1f-b71d-98bbab10a8df admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:05.571194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d682de0-e130-4b1f-b71d-98bbab10a8df admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:05.571194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d682de0-e130-4b1f-b71d-98bbab10a8df admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:05.571194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d682de0-e130-4b1f-b71d-98bbab10a8df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:05.571194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d682de0-e130-4b1f-b71d-98bbab10a8df admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:05.577715 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1186/2363] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:06:05.584050 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a8943-e2f2-40bc-b1a1-3739b5d6ae31 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:05.604550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5a8943-e2f2-40bc-b1a1-3739b5d6ae31 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:05.610034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418089f-aaf9-4083-9431-c0b35388fc98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:05.610706 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418089f-aaf9-4083-9431-c0b35388fc98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:05.611021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418089f-aaf9-4083-9431-c0b35388fc98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:05.611240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f418089f-aaf9-4083-9431-c0b35388fc98 admin admin] PATH_INFO: `/v3/projects/661496ac794348b2b758eb9f9ada11cf/users/d38b52c9c38149f09b6182bbc678896d/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:05.645115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418089f-aaf9-4083-9431-c0b35388fc98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=661496ac794348b2b758eb9f9ada11cf, user_id=d38b52c9c38149f09b6182bbc678896d, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:05.645115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f418089f-aaf9-4083-9431-c0b35388fc98 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:05.822318 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1178/2364] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:05 2022] PUT /identity/v3/projects/661496ac794348b2b758eb9f9ada11cf/users/d38b52c9c38149f09b6182bbc678896d/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:05.822907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552620d-f77d-41cc-94b8-226507415c50 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:05.822907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552620d-f77d-41cc-94b8-226507415c50 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:05.822907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552620d-f77d-41cc-94b8-226507415c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:05.830215 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418089f-aaf9-4083-9431-c0b35388fc98 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:05.867927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8552620d-f77d-41cc-94b8-226507415c50 None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:05.872669 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8552620d-f77d-41cc-94b8-226507415c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:06.029928 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f418089f-aaf9-4083-9431-c0b35388fc98 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:06.037200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1187/2365] 10.222.0.67 () {58 vars in 1009 bytes} [Mon Sep 26 18:06:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:06.053136 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4ad07-9060-4f66-bccc-152acff66db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:06.053801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4ad07-9060-4f66-bccc-152acff66db8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:06.054919 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4ad07-9060-4f66-bccc-152acff66db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:06.055879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd4ad07-9060-4f66-bccc-152acff66db8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:06.060511 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4ad07-9060-4f66-bccc-152acff66db8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:06.063028 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd4ad07-9060-4f66-bccc-152acff66db8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:06.071973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552620d-f77d-41cc-94b8-226507415c50 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:06.086990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1179/2366] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:05 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:06.093242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4ad07-9060-4f66-bccc-152acff66db8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:06.143490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd4ad07-9060-4f66-bccc-152acff66db8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:06.157934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6920b-6de1-4ae6-a3fc-d6131b6d9a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:06.159597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6920b-6de1-4ae6-a3fc-d6131b6d9a36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:06.160403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6920b-6de1-4ae6-a3fc-d6131b6d9a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:06.161023 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6920b-6de1-4ae6-a3fc-d6131b6d9a36 admin admin] PATH_INFO: `/v3/projects/661496ac794348b2b758eb9f9ada11cf/users/d38b52c9c38149f09b6182bbc678896d/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:06.164624 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae6920b-6de1-4ae6-a3fc-d6131b6d9a36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=661496ac794348b2b758eb9f9ada11cf, user_id=d38b52c9c38149f09b6182bbc678896d, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:06.169382 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae6920b-6de1-4ae6-a3fc-d6131b6d9a36 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:06.213352 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1180/2367] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:06 2022] PUT /identity/v3/projects/661496ac794348b2b758eb9f9ada11cf/users/d38b52c9c38149f09b6182bbc678896d/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:06.228184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f52f4f-5c54-44be-a5ea-c30a06809927 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:06.228833 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f52f4f-5c54-44be-a5ea-c30a06809927 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:06.229115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f52f4f-5c54-44be-a5ea-c30a06809927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:06.252985 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552620d-f77d-41cc-94b8-226507415c50 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:06.278140 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-93f52f4f-5c54-44be-a5ea-c30a06809927 None None] MFA Rules not processed for user `d38b52c9c38149f09b6182bbc678896d`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:06.280516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-93f52f4f-5c54-44be-a5ea-c30a06809927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:06.425606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54cfa17-5e65-4347-aaa6-0c27d7419f05 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:06.426837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cfa17-5e65-4347-aaa6-0c27d7419f05 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:06.427317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cfa17-5e65-4347-aaa6-0c27d7419f05 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:06.427876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54cfa17-5e65-4347-aaa6-0c27d7419f05 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:06.469236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1181/2368] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:06 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:06.482504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f52f4f-5c54-44be-a5ea-c30a06809927 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:06.484923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-93f52f4f-5c54-44be-a5ea-c30a06809927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:06.605828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54cfa17-5e65-4347-aaa6-0c27d7419f05 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:06.607970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54cfa17-5e65-4347-aaa6-0c27d7419f05 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:06.620565 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1188/2369] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:06:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:06:06.649062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f52f4f-5c54-44be-a5ea-c30a06809927 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:06.654783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307870de-c6a7-47e4-8d07-116b20bc34ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:06.655586 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307870de-c6a7-47e4-8d07-116b20bc34ff service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:06.657997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307870de-c6a7-47e4-8d07-116b20bc34ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:06.657997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307870de-c6a7-47e4-8d07-116b20bc34ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:06.768622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307870de-c6a7-47e4-8d07-116b20bc34ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:06.771920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307870de-c6a7-47e4-8d07-116b20bc34ff service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:06.789257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1182/2370] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:08.179384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fc4c9-07cb-4d8e-a5f4-db37ab11aef9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:08.180261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fc4c9-07cb-4d8e-a5f4-db37ab11aef9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:08.180261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689fc4c9-07cb-4d8e-a5f4-db37ab11aef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:08.258193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-689fc4c9-07cb-4d8e-a5f4-db37ab11aef9 None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:08.262044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-689fc4c9-07cb-4d8e-a5f4-db37ab11aef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:08.281563 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1189/2371] 10.222.0.67 () {58 vars in 967 bytes} [Mon Sep 26 18:06:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:08.305612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307870de-c6a7-47e4-8d07-116b20bc34ff service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:08.327167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307870de-c6a7-47e4-8d07-116b20bc34ff service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:08.334476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d110091-5837-48a2-9d06-dc16c6fc699a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:08.335799 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d110091-5837-48a2-9d06-dc16c6fc699a service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:08.335799 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d110091-5837-48a2-9d06-dc16c6fc699a service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:08.335799 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d110091-5837-48a2-9d06-dc16c6fc699a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:08.351613 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d110091-5837-48a2-9d06-dc16c6fc699a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:08.353738 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d110091-5837-48a2-9d06-dc16c6fc699a service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:08.375832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1183/2372] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:06:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:06:08.624901 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:09.330207 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:10.640318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689fc4c9-07cb-4d8e-a5f4-db37ab11aef9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:10.642515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-689fc4c9-07cb-4d8e-a5f4-db37ab11aef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:10.761609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689fc4c9-07cb-4d8e-a5f4-db37ab11aef9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:10.768427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc36d51-3762-447f-9009-7da34440fbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:10.769477 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc36d51-3762-447f-9009-7da34440fbdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:10.769694 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc36d51-3762-447f-9009-7da34440fbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:10.770105 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc36d51-3762-447f-9009-7da34440fbdf admin admin] PATH_INFO: `/v3/users/c742b7e0fac941a6ba58266bfc3e1440` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:10.772367 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc36d51-3762-447f-9009-7da34440fbdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=c742b7e0fac941a6ba58266bfc3e1440)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:10.776308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc36d51-3762-447f-9009-7da34440fbdf admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:11.150310 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9bc36d51-3762-447f-9009-7da34440fbdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c742b7e0fac941a6ba58266bfc3e1440'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:11.270878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9bc36d51-3762-447f-9009-7da34440fbdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c742b7e0fac941a6ba58266bfc3e1440'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:11.271949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9bc36d51-3762-447f-9009-7da34440fbdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c742b7e0fac941a6ba58266bfc3e1440'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:11.272986 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9bc36d51-3762-447f-9009-7da34440fbdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c742b7e0fac941a6ba58266bfc3e1440'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:11.298259 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9bc36d51-3762-447f-9009-7da34440fbdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c742b7e0fac941a6ba58266bfc3e1440'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:11.314192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1190/2373] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:10 2022] DELETE /identity/v3/users/c742b7e0fac941a6ba58266bfc3e1440 => generated 0 bytes in 675 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:11.323824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714a845-4765-421f-9f13-9ea01f103ffd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:11.323824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714a845-4765-421f-9f13-9ea01f103ffd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:11.323824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714a845-4765-421f-9f13-9ea01f103ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:11.349503 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e714a845-4765-421f-9f13-9ea01f103ffd None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:11.351626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e714a845-4765-421f-9f13-9ea01f103ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:11.470064 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1184/2374] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:11.489270 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc36d51-3762-447f-9009-7da34440fbdf admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:11.491004 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9bc36d51-3762-447f-9009-7da34440fbdf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:11.567238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc36d51-3762-447f-9009-7da34440fbdf admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:11.572060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca7f5e-86f5-4ecd-883a-88d9c4efd591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:11.572981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca7f5e-86f5-4ecd-883a-88d9c4efd591 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:11.573271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca7f5e-86f5-4ecd-883a-88d9c4efd591 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:11.573847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca7f5e-86f5-4ecd-883a-88d9c4efd591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:11.589286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca7f5e-86f5-4ecd-883a-88d9c4efd591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:11.592217 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca7f5e-86f5-4ecd-883a-88d9c4efd591 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:11.610834 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1191/2375] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:12.049358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714a845-4765-421f-9f13-9ea01f103ffd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:12.051753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e714a845-4765-421f-9f13-9ea01f103ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:12.087314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714a845-4765-421f-9f13-9ea01f103ffd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:12.108836 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdec950-2eff-4709-8b77-ef3f339eeb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:12.113931 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdec950-2eff-4709-8b77-ef3f339eeb94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:12.113931 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdec950-2eff-4709-8b77-ef3f339eeb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:12.113931 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdec950-2eff-4709-8b77-ef3f339eeb94 admin admin] PATH_INFO: `/v3/users/11e33b29101f4bcbb4bd204679f46d68` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:12.116476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdec950-2eff-4709-8b77-ef3f339eeb94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e33b29101f4bcbb4bd204679f46d68)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:12.120209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdec950-2eff-4709-8b77-ef3f339eeb94 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:12.134877 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca7f5e-86f5-4ecd-883a-88d9c4efd591 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:12.137153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a6ca7f5e-86f5-4ecd-883a-88d9c4efd591 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:12.172943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca7f5e-86f5-4ecd-883a-88d9c4efd591 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:12.184610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8917cf7-741c-4730-8063-da573ad735e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:12.185628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8917cf7-741c-4730-8063-da573ad735e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:12.186170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8917cf7-741c-4730-8063-da573ad735e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:12.186561 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8917cf7-741c-4730-8063-da573ad735e8 admin admin] PATH_INFO: `/v3/users/046e50a17a6a4783895782d3aa2172e2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:12.188766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8917cf7-741c-4730-8063-da573ad735e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=046e50a17a6a4783895782d3aa2172e2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:12.191773 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8917cf7-741c-4730-8063-da573ad735e8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:12.451897 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ebdec950-2eff-4709-8b77-ef3f339eeb94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e33b29101f4bcbb4bd204679f46d68'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.615720 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c8917cf7-741c-4730-8063-da573ad735e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '046e50a17a6a4783895782d3aa2172e2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.682714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ebdec950-2eff-4709-8b77-ef3f339eeb94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e33b29101f4bcbb4bd204679f46d68'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.683263 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ebdec950-2eff-4709-8b77-ef3f339eeb94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e33b29101f4bcbb4bd204679f46d68'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.684333 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ebdec950-2eff-4709-8b77-ef3f339eeb94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e33b29101f4bcbb4bd204679f46d68'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.694546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ebdec950-2eff-4709-8b77-ef3f339eeb94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e33b29101f4bcbb4bd204679f46d68'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.720643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1185/2376] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:12 2022] DELETE /identity/v3/users/11e33b29101f4bcbb4bd204679f46d68 => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:12.720643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdec950-2eff-4709-8b77-ef3f339eeb94 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:12.720643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ebdec950-2eff-4709-8b77-ef3f339eeb94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:12.823027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdec950-2eff-4709-8b77-ef3f339eeb94 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:12.831115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7c785-8816-4dd7-9729-5dcf8ae505d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:12.832110 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7c785-8816-4dd7-9729-5dcf8ae505d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:12.832552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7c785-8816-4dd7-9729-5dcf8ae505d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:12.833180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a7c785-8816-4dd7-9729-5dcf8ae505d8 admin admin] PATH_INFO: `/v3/projects/762f4bf5996e4da99f2a9bce719999c9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:12.834782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7c785-8816-4dd7-9729-5dcf8ae505d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=762f4bf5996e4da99f2a9bce719999c9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:12.844381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a7c785-8816-4dd7-9729-5dcf8ae505d8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:12.860906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c8917cf7-741c-4730-8063-da573ad735e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '046e50a17a6a4783895782d3aa2172e2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.864791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c8917cf7-741c-4730-8063-da573ad735e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '046e50a17a6a4783895782d3aa2172e2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.864791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c8917cf7-741c-4730-8063-da573ad735e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '046e50a17a6a4783895782d3aa2172e2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.896242 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c8917cf7-741c-4730-8063-da573ad735e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '046e50a17a6a4783895782d3aa2172e2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:12.919781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1192/2377] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:12 2022] DELETE /identity/v3/users/046e50a17a6a4783895782d3aa2172e2 => generated 0 bytes in 781 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:12.920932 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c1d39-3b09-48ce-af67-06ae4e59c641 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:12.921338 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c1d39-3b09-48ce-af67-06ae4e59c641 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:12.922673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c1d39-3b09-48ce-af67-06ae4e59c641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:12.950575 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-183c1d39-3b09-48ce-af67-06ae4e59c641 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:12.952044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-183c1d39-3b09-48ce-af67-06ae4e59c641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:13.064142 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-38a7c785-8816-4dd7-9729-5dcf8ae505d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '762f4bf5996e4da99f2a9bce719999c9', 'request_id': 'req-38a7c785-8816-4dd7-9729-5dcf8ae505d8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:13.076619 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-38a7c785-8816-4dd7-9729-5dcf8ae505d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '762f4bf5996e4da99f2a9bce719999c9', 'request_id': 'req-38a7c785-8816-4dd7-9729-5dcf8ae505d8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:13.090942 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1193/2378] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:13.094359 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215b4ac-1ec5-4d37-bec4-82576a58a723 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.095016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215b4ac-1ec5-4d37-bec4-82576a58a723 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.095620 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0215b4ac-1ec5-4d37-bec4-82576a58a723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.147742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0215b4ac-1ec5-4d37-bec4-82576a58a723 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:13.151358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0215b4ac-1ec5-4d37-bec4-82576a58a723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:13.240647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-38a7c785-8816-4dd7-9729-5dcf8ae505d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '762f4bf5996e4da99f2a9bce719999c9', 'request_id': 'req-38a7c785-8816-4dd7-9729-5dcf8ae505d8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:13.258711 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-38a7c785-8816-4dd7-9729-5dcf8ae505d8 admin admin] The token cache is being invalidate because project 762f4bf5996e4da99f2a9bce719999c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:13.259053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-38a7c785-8816-4dd7-9729-5dcf8ae505d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:13.260130 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-38a7c785-8816-4dd7-9729-5dcf8ae505d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:13.261620 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1194/2379] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:13.262189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1186/2380] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:12 2022] DELETE /identity/v3/projects/762f4bf5996e4da99f2a9bce719999c9 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:13.263468 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7c785-8816-4dd7-9729-5dcf8ae505d8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:13.269855 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-38a7c785-8816-4dd7-9729-5dcf8ae505d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:13.270620 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df380f-a4a6-429a-9dd9-29c8aba12f07 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.271051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df380f-a4a6-429a-9dd9-29c8aba12f07 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.271597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23df380f-a4a6-429a-9dd9-29c8aba12f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.303914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-23df380f-a4a6-429a-9dd9-29c8aba12f07 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:13.305382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-23df380f-a4a6-429a-9dd9-29c8aba12f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:13.335798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1195/2381] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:13.335798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df380f-a4a6-429a-9dd9-29c8aba12f07 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:13.335798 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-23df380f-a4a6-429a-9dd9-29c8aba12f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:13.422907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a7c785-8816-4dd7-9729-5dcf8ae505d8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:13.431096 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00d2b6-7519-4813-be5e-8ee8e021e548 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:13.432603 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00d2b6-7519-4813-be5e-8ee8e021e548 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.444470 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00d2b6-7519-4813-be5e-8ee8e021e548 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.444470 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00d2b6-7519-4813-be5e-8ee8e021e548 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.444470 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4c00d2b6-7519-4813-be5e-8ee8e021e548 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:13.458220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23df380f-a4a6-429a-9dd9-29c8aba12f07 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:13.466081 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b022e2c-2b9c-40ed-9974-39fadbc87136 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:13.467568 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b022e2c-2b9c-40ed-9974-39fadbc87136 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.468377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b022e2c-2b9c-40ed-9974-39fadbc87136 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.469590 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b022e2c-2b9c-40ed-9974-39fadbc87136 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.473737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9b022e2c-2b9c-40ed-9974-39fadbc87136 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:13.482548 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00d2b6-7519-4813-be5e-8ee8e021e548 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:13.486654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00d2b6-7519-4813-be5e-8ee8e021e548 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:13.500307 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b022e2c-2b9c-40ed-9974-39fadbc87136 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:13.504121 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b022e2c-2b9c-40ed-9974-39fadbc87136 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:13.507676 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1187/2382] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:13.510986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894e16d-3726-49fd-8b6f-e5d81142b800 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.512105 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894e16d-3726-49fd-8b6f-e5d81142b800 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.513032 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9894e16d-3726-49fd-8b6f-e5d81142b800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.527372 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1196/2383] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:13.561511 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9894e16d-3726-49fd-8b6f-e5d81142b800 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:13.565373 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9894e16d-3726-49fd-8b6f-e5d81142b800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:13.582084 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1188/2384] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:13.589682 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b022e2c-2b9c-40ed-9974-39fadbc87136 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:13.605777 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b022e2c-2b9c-40ed-9974-39fadbc87136 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:13.615082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b61e5-e25d-4f5e-932e-14d8e7ffd7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:13.617426 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b61e5-e25d-4f5e-932e-14d8e7ffd7c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.617703 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b61e5-e25d-4f5e-932e-14d8e7ffd7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.618006 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b61e5-e25d-4f5e-932e-14d8e7ffd7c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.618761 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b61e5-e25d-4f5e-932e-14d8e7ffd7c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:06:13.619273 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b61e5-e25d-4f5e-932e-14d8e7ffd7c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:13.623531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b61e5-e25d-4f5e-932e-14d8e7ffd7c7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:13.654822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1197/2385] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:06:13.655332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894e16d-3726-49fd-8b6f-e5d81142b800 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:13.694873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9894e16d-3726-49fd-8b6f-e5d81142b800 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:13.722199 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4cc5e-3a3d-4b8f-8d36-e982c64806fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:13.725644 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4cc5e-3a3d-4b8f-8d36-e982c64806fa admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.730356 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4cc5e-3a3d-4b8f-8d36-e982c64806fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.732130 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a4cc5e-3a3d-4b8f-8d36-e982c64806fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.736031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4cc5e-3a3d-4b8f-8d36-e982c64806fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:13.745487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a4cc5e-3a3d-4b8f-8d36-e982c64806fa admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:13.811966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b61e5-e25d-4f5e-932e-14d8e7ffd7c7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:13.815027 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-db5b61e5-e25d-4f5e-932e-14d8e7ffd7c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:13.841338 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b61e5-e25d-4f5e-932e-14d8e7ffd7c7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:13.850021 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9210c1-dcf4-4e88-b25e-b6b4c3a844c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:13.850966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9210c1-dcf4-4e88-b25e-b6b4c3a844c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:13.851508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9210c1-dcf4-4e88-b25e-b6b4c3a844c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:13.851937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9210c1-dcf4-4e88-b25e-b6b4c3a844c9 admin admin] PATH_INFO: `/v3/users/5a91b3b4d37d4039b85e655132a12e08` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:13.854822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9210c1-dcf4-4e88-b25e-b6b4c3a844c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a91b3b4d37d4039b85e655132a12e08)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:13.856874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9210c1-dcf4-4e88-b25e-b6b4c3a844c9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:13.941173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1189/2386] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:13 2022] POST /identity/v3/projects => generated 371 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:13.943416 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4cc5e-3a3d-4b8f-8d36-e982c64806fa admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:14.071543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a4cc5e-3a3d-4b8f-8d36-e982c64806fa admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:14.079073 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6be9b-7757-4c19-b11d-a616b1347577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:14.080306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6be9b-7757-4c19-b11d-a616b1347577 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:14.080876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6be9b-7757-4c19-b11d-a616b1347577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:14.081370 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6be9b-7757-4c19-b11d-a616b1347577 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:14.082736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6be9b-7757-4c19-b11d-a616b1347577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:14.088832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6be9b-7757-4c19-b11d-a616b1347577 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:14.271817 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1190/2387] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:13 2022] POST /identity/v3/users => generated 286 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:14.271817 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6be9b-7757-4c19-b11d-a616b1347577 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:14.304269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ae9210c1-dcf4-4e88-b25e-b6b4c3a844c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a91b3b4d37d4039b85e655132a12e08'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:14.307572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6be9b-7757-4c19-b11d-a616b1347577 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:14.323627 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade8a66-3756-4e61-b0dd-e3ce8cb994e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:14.324884 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade8a66-3756-4e61-b0dd-e3ce8cb994e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:14.325468 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade8a66-3756-4e61-b0dd-e3ce8cb994e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:14.327801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ade8a66-3756-4e61-b0dd-e3ce8cb994e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:14.334618 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade8a66-3756-4e61-b0dd-e3ce8cb994e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:14.340145 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ade8a66-3756-4e61-b0dd-e3ce8cb994e2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:14.369585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1191/2388] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:06:14.379138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade8a66-3756-4e61-b0dd-e3ce8cb994e2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:14.396652 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ae9210c1-dcf4-4e88-b25e-b6b4c3a844c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a91b3b4d37d4039b85e655132a12e08'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:14.397869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ae9210c1-dcf4-4e88-b25e-b6b4c3a844c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a91b3b4d37d4039b85e655132a12e08'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:14.398913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ae9210c1-dcf4-4e88-b25e-b6b4c3a844c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a91b3b4d37d4039b85e655132a12e08'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:14.405862 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ade8a66-3756-4e61-b0dd-e3ce8cb994e2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:14.420026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3512d1-68b9-43e8-a838-7a3eae1a95f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:14.420808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3512d1-68b9-43e8-a838-7a3eae1a95f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:14.421549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3512d1-68b9-43e8-a838-7a3eae1a95f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:14.421897 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3512d1-68b9-43e8-a838-7a3eae1a95f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:14.422610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3512d1-68b9-43e8-a838-7a3eae1a95f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:14.428548 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3512d1-68b9-43e8-a838-7a3eae1a95f6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:14.444508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ae9210c1-dcf4-4e88-b25e-b6b4c3a844c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a91b3b4d37d4039b85e655132a12e08'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:14.460945 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1192/2389] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:14 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:06:14.462240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3512d1-68b9-43e8-a838-7a3eae1a95f6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:14.465851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8e3512d1-68b9-43e8-a838-7a3eae1a95f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:14.483272 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1198/2390] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:13 2022] DELETE /identity/v3/users/5a91b3b4d37d4039b85e655132a12e08 => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:14.490485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70995c1b-2d05-4c45-8148-3b81c519f701 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:14.491478 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70995c1b-2d05-4c45-8148-3b81c519f701 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:14.495200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70995c1b-2d05-4c45-8148-3b81c519f701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:14.500331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3512d1-68b9-43e8-a838-7a3eae1a95f6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:14.547959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-70995c1b-2d05-4c45-8148-3b81c519f701 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:14.551531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-70995c1b-2d05-4c45-8148-3b81c519f701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:14.775715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790080f9-6e6c-4bd8-9eec-534c420fdc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:14.789046 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790080f9-6e6c-4bd8-9eec-534c420fdc9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:14.789046 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790080f9-6e6c-4bd8-9eec-534c420fdc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:14.789046 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790080f9-6e6c-4bd8-9eec-534c420fdc9d admin admin] PATH_INFO: `/v3/projects/08e4bf78b8f043a598a4b6a1d8acc219/users/183fe84502664f49a701a5e9eb4f6cd7/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:14.789995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1199/2391] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:14.821265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790080f9-6e6c-4bd8-9eec-534c420fdc9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=08e4bf78b8f043a598a4b6a1d8acc219, user_id=183fe84502664f49a701a5e9eb4f6cd7, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:14.826888 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790080f9-6e6c-4bd8-9eec-534c420fdc9d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:14.838253 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70995c1b-2d05-4c45-8148-3b81c519f701 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:14.841552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-70995c1b-2d05-4c45-8148-3b81c519f701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:14.950592 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70995c1b-2d05-4c45-8148-3b81c519f701 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:14.957337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2e324-4471-456c-8763-cc2981c6c4bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:14.957861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2e324-4471-456c-8763-cc2981c6c4bd service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:14.958867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2e324-4471-456c-8763-cc2981c6c4bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:14.959221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c2e324-4471-456c-8763-cc2981c6c4bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:14.971030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2e324-4471-456c-8763-cc2981c6c4bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:14.974051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c2e324-4471-456c-8763-cc2981c6c4bd service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:14.997896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1200/2392] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:15.022652 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2e324-4471-456c-8763-cc2981c6c4bd service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:15.025624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-27c2e324-4471-456c-8763-cc2981c6c4bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:15.055164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c2e324-4471-456c-8763-cc2981c6c4bd service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:15.080254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:15.082302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:15.083028 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:15.085060 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1193/2393] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:14 2022] PUT /identity/v3/projects/08e4bf78b8f043a598a4b6a1d8acc219/users/183fe84502664f49a701a5e9eb4f6cd7/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:15.094713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c admin admin] PATH_INFO: `/v3/projects/dee85100e2744253bece8a0d74c79f41` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:15.094713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c admin admin] RBAC: Authorizing `identity:delete_project(project_id=dee85100e2744253bece8a0d74c79f41)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:15.096425 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:15.098922 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be9427-1b20-4dc7-9a6e-a4911c897f6d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:15.099333 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be9427-1b20-4dc7-9a6e-a4911c897f6d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:15.099512 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be9427-1b20-4dc7-9a6e-a4911c897f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:15.134253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-03be9427-1b20-4dc7-9a6e-a4911c897f6d None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:15.135899 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-03be9427-1b20-4dc7-9a6e-a4911c897f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:15.284452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1194/2394] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:15.290154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be9427-1b20-4dc7-9a6e-a4911c897f6d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:15.292701 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-03be9427-1b20-4dc7-9a6e-a4911c897f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:15.314150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be9427-1b20-4dc7-9a6e-a4911c897f6d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:15.320621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeed112-3d19-45f8-bf3b-ed41359e7f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:15.321469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeed112-3d19-45f8-bf3b-ed41359e7f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:15.321918 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeed112-3d19-45f8-bf3b-ed41359e7f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:15.322481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeed112-3d19-45f8-bf3b-ed41359e7f71 admin admin] PATH_INFO: `/v3/projects/1e9d16b034454af791f2d947bc22aff6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:15.324685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeed112-3d19-45f8-bf3b-ed41359e7f71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e9d16b034454af791f2d947bc22aff6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:15.327043 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeed112-3d19-45f8-bf3b-ed41359e7f71 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:15.460364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dee85100e2744253bece8a0d74c79f41', 'request_id': 'req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:15.470731 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dee85100e2744253bece8a0d74c79f41', 'request_id': 'req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:15.686754 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dee85100e2744253bece8a0d74c79f41', 'request_id': 'req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:15.716889 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c admin admin] The token cache is being invalidate because project dee85100e2744253bece8a0d74c79f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:15.719181 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:15.721465 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:15.726956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1201/2395] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:15 2022] DELETE /identity/v3/projects/dee85100e2744253bece8a0d74c79f41 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:15.728221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:15.732377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:15.737016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-caeed112-3d19-45f8-bf3b-ed41359e7f71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e9d16b034454af791f2d947bc22aff6', 'request_id': 'req-caeed112-3d19-45f8-bf3b-ed41359e7f71'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:15.770572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-caeed112-3d19-45f8-bf3b-ed41359e7f71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e9d16b034454af791f2d947bc22aff6', 'request_id': 'req-caeed112-3d19-45f8-bf3b-ed41359e7f71'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:15.944767 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86499e4-9bb5-4cc9-af4c-6468d5cdd09c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:16.078148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c69be8-3348-4b88-84f2-64fe34cb2ba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:16.079486 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c69be8-3348-4b88-84f2-64fe34cb2ba9 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:16.084491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c69be8-3348-4b88-84f2-64fe34cb2ba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:16.084951 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c69be8-3348-4b88-84f2-64fe34cb2ba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:16.086282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-caeed112-3d19-45f8-bf3b-ed41359e7f71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e9d16b034454af791f2d947bc22aff6', 'request_id': 'req-caeed112-3d19-45f8-bf3b-ed41359e7f71'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:16.089115 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-02c69be8-3348-4b88-84f2-64fe34cb2ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:16.111019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-caeed112-3d19-45f8-bf3b-ed41359e7f71 admin admin] The token cache is being invalidate because project 1e9d16b034454af791f2d947bc22aff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:16.111284 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-caeed112-3d19-45f8-bf3b-ed41359e7f71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:16.112097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-caeed112-3d19-45f8-bf3b-ed41359e7f71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:16.113748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1195/2396] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:15 2022] DELETE /identity/v3/projects/1e9d16b034454af791f2d947bc22aff6 => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:16.118080 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeed112-3d19-45f8-bf3b-ed41359e7f71 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:16.119213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:16.119983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-caeed112-3d19-45f8-bf3b-ed41359e7f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:16.281622 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c69be8-3348-4b88-84f2-64fe34cb2ba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:16.284689 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c69be8-3348-4b88-84f2-64fe34cb2ba9 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:16.320322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1202/2397] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:16.327610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeed112-3d19-45f8-bf3b-ed41359e7f71 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:16.335909 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604ff23-6df4-4776-852a-ed1bcbdc10fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:16.337163 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604ff23-6df4-4776-852a-ed1bcbdc10fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:16.337620 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604ff23-6df4-4776-852a-ed1bcbdc10fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:16.338031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6604ff23-6df4-4776-852a-ed1bcbdc10fa admin admin] PATH_INFO: `/v3/users/d38b52c9c38149f09b6182bbc678896d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:16.340004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604ff23-6df4-4776-852a-ed1bcbdc10fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=d38b52c9c38149f09b6182bbc678896d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:16.342400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6604ff23-6df4-4776-852a-ed1bcbdc10fa admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:16.404394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:16.834104 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6604ff23-6df4-4776-852a-ed1bcbdc10fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd38b52c9c38149f09b6182bbc678896d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:16.916926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c69be8-3348-4b88-84f2-64fe34cb2ba9 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:16.916926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-02c69be8-3348-4b88-84f2-64fe34cb2ba9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:16.952543 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c69be8-3348-4b88-84f2-64fe34cb2ba9 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:16.963915 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382dba3-c33a-48a6-b681-225ce1a93b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:16.965913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382dba3-c33a-48a6-b681-225ce1a93b3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:16.966592 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382dba3-c33a-48a6-b681-225ce1a93b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:16.967094 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382dba3-c33a-48a6-b681-225ce1a93b3b admin admin] PATH_INFO: `/v3/projects/9a38597d4f364326b91828b5786cfe49` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:16.969579 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382dba3-c33a-48a6-b681-225ce1a93b3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a38597d4f364326b91828b5786cfe49)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:16.973434 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382dba3-c33a-48a6-b681-225ce1a93b3b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:17.379933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6604ff23-6df4-4776-852a-ed1bcbdc10fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd38b52c9c38149f09b6182bbc678896d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:17.380983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6604ff23-6df4-4776-852a-ed1bcbdc10fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd38b52c9c38149f09b6182bbc678896d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:17.382242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6604ff23-6df4-4776-852a-ed1bcbdc10fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd38b52c9c38149f09b6182bbc678896d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:17.401223 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6604ff23-6df4-4776-852a-ed1bcbdc10fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd38b52c9c38149f09b6182bbc678896d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:17.436429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1196/2398] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:16 2022] DELETE /identity/v3/users/d38b52c9c38149f09b6182bbc678896d => generated 0 bytes in 1319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:17.546222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8382dba3-c33a-48a6-b681-225ce1a93b3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a38597d4f364326b91828b5786cfe49', 'request_id': 'req-8382dba3-c33a-48a6-b681-225ce1a93b3b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:17.596554 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8382dba3-c33a-48a6-b681-225ce1a93b3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a38597d4f364326b91828b5786cfe49', 'request_id': 'req-8382dba3-c33a-48a6-b681-225ce1a93b3b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:17.900380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8382dba3-c33a-48a6-b681-225ce1a93b3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a38597d4f364326b91828b5786cfe49', 'request_id': 'req-8382dba3-c33a-48a6-b681-225ce1a93b3b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:17.943819 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8382dba3-c33a-48a6-b681-225ce1a93b3b admin admin] The token cache is being invalidate because project 9a38597d4f364326b91828b5786cfe49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:17.944819 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8382dba3-c33a-48a6-b681-225ce1a93b3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:17.967193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8382dba3-c33a-48a6-b681-225ce1a93b3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:17.970914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1203/2399] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:16 2022] DELETE /identity/v3/projects/9a38597d4f364326b91828b5786cfe49 => generated 0 bytes in 1078 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:17.976813 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604ff23-6df4-4776-852a-ed1bcbdc10fa admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:17.985841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6604ff23-6df4-4776-852a-ed1bcbdc10fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:18.115636 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6604ff23-6df4-4776-852a-ed1bcbdc10fa admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:18.272332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e284c-cddf-4e60-9a20-8554a2f7cac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:18.273421 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e284c-cddf-4e60-9a20-8554a2f7cac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:18.273909 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e284c-cddf-4e60-9a20-8554a2f7cac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:18.274676 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e284c-cddf-4e60-9a20-8554a2f7cac6 admin admin] PATH_INFO: `/v3/users/a2a45c7ee8234cfba1dd273fb5682177` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:18.278013 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e284c-cddf-4e60-9a20-8554a2f7cac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2a45c7ee8234cfba1dd273fb5682177)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:18.282056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e284c-cddf-4e60-9a20-8554a2f7cac6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:18.619380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382dba3-c33a-48a6-b681-225ce1a93b3b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:18.621454 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8382dba3-c33a-48a6-b681-225ce1a93b3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:18.651148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382dba3-c33a-48a6-b681-225ce1a93b3b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:18.660558 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56155f77-679a-41a8-8ec1-dc1075a83785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:18.661592 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56155f77-679a-41a8-8ec1-dc1075a83785 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:18.662071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56155f77-679a-41a8-8ec1-dc1075a83785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:18.662707 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56155f77-679a-41a8-8ec1-dc1075a83785 admin admin] PATH_INFO: `/v3/projects/661496ac794348b2b758eb9f9ada11cf` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:18.664969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56155f77-679a-41a8-8ec1-dc1075a83785 admin admin] RBAC: Authorizing `identity:delete_project(project_id=661496ac794348b2b758eb9f9ada11cf)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:18.667494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56155f77-679a-41a8-8ec1-dc1075a83785 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:19.180261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56155f77-679a-41a8-8ec1-dc1075a83785 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '661496ac794348b2b758eb9f9ada11cf', 'request_id': 'req-56155f77-679a-41a8-8ec1-dc1075a83785'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:19.180989 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-259e284c-cddf-4e60-9a20-8554a2f7cac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2a45c7ee8234cfba1dd273fb5682177'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:19.192439 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56155f77-679a-41a8-8ec1-dc1075a83785 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '661496ac794348b2b758eb9f9ada11cf', 'request_id': 'req-56155f77-679a-41a8-8ec1-dc1075a83785'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:19.471911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-259e284c-cddf-4e60-9a20-8554a2f7cac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2a45c7ee8234cfba1dd273fb5682177'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:19.472651 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-259e284c-cddf-4e60-9a20-8554a2f7cac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2a45c7ee8234cfba1dd273fb5682177'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:19.473643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-259e284c-cddf-4e60-9a20-8554a2f7cac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2a45c7ee8234cfba1dd273fb5682177'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:19.487394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-259e284c-cddf-4e60-9a20-8554a2f7cac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2a45c7ee8234cfba1dd273fb5682177'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:19.529137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1197/2400] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:17 2022] DELETE /identity/v3/users/a2a45c7ee8234cfba1dd273fb5682177 => generated 0 bytes in 1548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:19.597055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56155f77-679a-41a8-8ec1-dc1075a83785 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '661496ac794348b2b758eb9f9ada11cf', 'request_id': 'req-56155f77-679a-41a8-8ec1-dc1075a83785'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:19.642489 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56155f77-679a-41a8-8ec1-dc1075a83785 admin admin] The token cache is being invalidate because project 661496ac794348b2b758eb9f9ada11cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:19.643013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56155f77-679a-41a8-8ec1-dc1075a83785 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:19.645721 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56155f77-679a-41a8-8ec1-dc1075a83785 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:19.651713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1204/2401] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:18 2022] DELETE /identity/v3/projects/661496ac794348b2b758eb9f9ada11cf => generated 0 bytes in 1034 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:19.657851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06510e-ffe0-41bc-9533-93398207100c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:19.659047 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06510e-ffe0-41bc-9533-93398207100c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:19.659327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c06510e-ffe0-41bc-9533-93398207100c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:19.700350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9c06510e-ffe0-41bc-9533-93398207100c None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:19.702424 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9c06510e-ffe0-41bc-9533-93398207100c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:19.786903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1198/2402] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:19.796501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7431b-78bb-46c1-9306-b12ab9108120 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:19.797147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7431b-78bb-46c1-9306-b12ab9108120 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:19.797300 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7431b-78bb-46c1-9306-b12ab9108120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:19.823701 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bdc7431b-78bb-46c1-9306-b12ab9108120 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:19.825247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bdc7431b-78bb-46c1-9306-b12ab9108120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:19.837941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1205/2403] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:19.842745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06510e-ffe0-41bc-9533-93398207100c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:19.864034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c06510e-ffe0-41bc-9533-93398207100c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:19.869176 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0ba7d-2dc9-4318-a992-534dddd949a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:19.869743 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0ba7d-2dc9-4318-a992-534dddd949a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:19.870026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0ba7d-2dc9-4318-a992-534dddd949a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:19.870306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a0ba7d-2dc9-4318-a992-534dddd949a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:19.871004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0ba7d-2dc9-4318-a992-534dddd949a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:06:19.871305 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0ba7d-2dc9-4318-a992-534dddd949a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:19.873662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a0ba7d-2dc9-4318-a992-534dddd949a2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:19.894753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1199/2404] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:06:19.903898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7431b-78bb-46c1-9306-b12ab9108120 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:19.919880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7431b-78bb-46c1-9306-b12ab9108120 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:19.928752 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae554bc-4971-427e-9b75-8f85887bdccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:19.929546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae554bc-4971-427e-9b75-8f85887bdccb admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:19.930101 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae554bc-4971-427e-9b75-8f85887bdccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:19.930776 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae554bc-4971-427e-9b75-8f85887bdccb admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:19.931701 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae554bc-4971-427e-9b75-8f85887bdccb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:19.934843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae554bc-4971-427e-9b75-8f85887bdccb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:20.341561 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1206/2405] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:19 2022] POST /identity/v3/projects => generated 391 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:06:20.357161 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0ba7d-2dc9-4318-a992-534dddd949a2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:20.373855 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a0ba7d-2dc9-4318-a992-534dddd949a2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:20.490481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e71ed8-50bf-406a-b4a7-bc6b789b1f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:20.490909 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e71ed8-50bf-406a-b4a7-bc6b789b1f7a admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.491265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e71ed8-50bf-406a-b4a7-bc6b789b1f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.491428 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e71ed8-50bf-406a-b4a7-bc6b789b1f7a admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.492241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e71ed8-50bf-406a-b4a7-bc6b789b1f7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:20.494537 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e71ed8-50bf-406a-b4a7-bc6b789b1f7a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:20.708764 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1200/2406] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:20 2022] POST /identity/v3/users => generated 296 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:20.713258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae554bc-4971-427e-9b75-8f85887bdccb admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:20.733600 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae554bc-4971-427e-9b75-8f85887bdccb admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:20.742533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179027c-c009-4df5-8272-4ba339e659d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:20.744105 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179027c-c009-4df5-8272-4ba339e659d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.744790 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179027c-c009-4df5-8272-4ba339e659d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.745286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179027c-c009-4df5-8272-4ba339e659d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.746523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179027c-c009-4df5-8272-4ba339e659d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:20.749699 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179027c-c009-4df5-8272-4ba339e659d3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:20.770838 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1207/2407] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:06:20.781213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e71ed8-50bf-406a-b4a7-bc6b789b1f7a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:20.787216 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e71ed8-50bf-406a-b4a7-bc6b789b1f7a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:20.793077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26b18a-2819-4cc2-9fb8-060fcd912bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:20.794000 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26b18a-2819-4cc2-9fb8-060fcd912bbe admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.794475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26b18a-2819-4cc2-9fb8-060fcd912bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.794913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d26b18a-2819-4cc2-9fb8-060fcd912bbe admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.796150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26b18a-2819-4cc2-9fb8-060fcd912bbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:20.799217 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d26b18a-2819-4cc2-9fb8-060fcd912bbe admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:20.810574 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1201/2408] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:20.819770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179027c-c009-4df5-8272-4ba339e659d3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:20.838756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179027c-c009-4df5-8272-4ba339e659d3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:20.846104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e9650-13c6-4bbb-b2d6-1eb70235bc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:20.846849 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e9650-13c6-4bbb-b2d6-1eb70235bc44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.847130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e9650-13c6-4bbb-b2d6-1eb70235bc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.847416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e9650-13c6-4bbb-b2d6-1eb70235bc44 admin admin] PATH_INFO: `/v3/projects/b62230b4cd3943c5bf03d271d83af6a5/users/0f7418db7d8948ec8a165793fd99c406/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.871255 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e9650-13c6-4bbb-b2d6-1eb70235bc44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b62230b4cd3943c5bf03d271d83af6a5, user_id=0f7418db7d8948ec8a165793fd99c406, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:20.876334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e9650-13c6-4bbb-b2d6-1eb70235bc44 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:20.912037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26b18a-2819-4cc2-9fb8-060fcd912bbe admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:20.914667 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9d26b18a-2819-4cc2-9fb8-060fcd912bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:20.942444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d26b18a-2819-4cc2-9fb8-060fcd912bbe admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:20.954475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853063e6-c2c6-4ff6-866c-615146e362d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:20.955497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853063e6-c2c6-4ff6-866c-615146e362d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:20.955892 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853063e6-c2c6-4ff6-866c-615146e362d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:20.959180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853063e6-c2c6-4ff6-866c-615146e362d9 admin admin] PATH_INFO: `/v3/projects/89d41f3248a84f169b8240fdfa62ef92` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:20.959180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853063e6-c2c6-4ff6-866c-615146e362d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89d41f3248a84f169b8240fdfa62ef92)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:20.963924 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853063e6-c2c6-4ff6-866c-615146e362d9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:21.016967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1208/2409] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:20 2022] PUT /identity/v3/projects/b62230b4cd3943c5bf03d271d83af6a5/users/0f7418db7d8948ec8a165793fd99c406/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:21.037486 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69058b-a4db-48d5-8756-0e4a90fa4eaf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:21.042154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69058b-a4db-48d5-8756-0e4a90fa4eaf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:21.042154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf69058b-a4db-48d5-8756-0e4a90fa4eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:21.127559 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bf69058b-a4db-48d5-8756-0e4a90fa4eaf None None] MFA Rules not processed for user `0f7418db7d8948ec8a165793fd99c406`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:21.129786 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bf69058b-a4db-48d5-8756-0e4a90fa4eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:21.244320 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1209/2410] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:21 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:21.261268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69058b-a4db-48d5-8756-0e4a90fa4eaf None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:21.263352 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bf69058b-a4db-48d5-8756-0e4a90fa4eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:21.324838 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-853063e6-c2c6-4ff6-866c-615146e362d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89d41f3248a84f169b8240fdfa62ef92', 'request_id': 'req-853063e6-c2c6-4ff6-866c-615146e362d9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:21.342074 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-853063e6-c2c6-4ff6-866c-615146e362d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89d41f3248a84f169b8240fdfa62ef92', 'request_id': 'req-853063e6-c2c6-4ff6-866c-615146e362d9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:21.350148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf69058b-a4db-48d5-8756-0e4a90fa4eaf None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:21.431868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba90048-0bca-4f3d-98be-a600fef8ee8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:21.432668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba90048-0bca-4f3d-98be-a600fef8ee8a service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:21.433040 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba90048-0bca-4f3d-98be-a600fef8ee8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:21.433403 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba90048-0bca-4f3d-98be-a600fef8ee8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:21.519292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba90048-0bca-4f3d-98be-a600fef8ee8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:21.523974 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba90048-0bca-4f3d-98be-a600fef8ee8a service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:21.532766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1210/2411] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:21 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:06:21.604973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-853063e6-c2c6-4ff6-866c-615146e362d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89d41f3248a84f169b8240fdfa62ef92', 'request_id': 'req-853063e6-c2c6-4ff6-866c-615146e362d9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:21.616931 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-853063e6-c2c6-4ff6-866c-615146e362d9 admin admin] The token cache is being invalidate because project 89d41f3248a84f169b8240fdfa62ef92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:21.617229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-853063e6-c2c6-4ff6-866c-615146e362d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:21.617910 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-853063e6-c2c6-4ff6-866c-615146e362d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:21.622572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1202/2412] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:20 2022] DELETE /identity/v3/projects/89d41f3248a84f169b8240fdfa62ef92 => generated 0 bytes in 709 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:21.624245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba90048-0bca-4f3d-98be-a600fef8ee8a service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:21.755127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba90048-0bca-4f3d-98be-a600fef8ee8a service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:21.764307 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782b4ae-a005-4186-b2e7-5dcdc75f985a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:21.765482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782b4ae-a005-4186-b2e7-5dcdc75f985a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:21.766251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782b4ae-a005-4186-b2e7-5dcdc75f985a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:21.766733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782b4ae-a005-4186-b2e7-5dcdc75f985a admin admin] PATH_INFO: `/v3/users/ef3744018d1845a08a540f40db6b08b5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:21.769096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782b4ae-a005-4186-b2e7-5dcdc75f985a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef3744018d1845a08a540f40db6b08b5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:21.773168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9782b4ae-a005-4186-b2e7-5dcdc75f985a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:22.327678 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9782b4ae-a005-4186-b2e7-5dcdc75f985a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef3744018d1845a08a540f40db6b08b5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:22.522670 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9782b4ae-a005-4186-b2e7-5dcdc75f985a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef3744018d1845a08a540f40db6b08b5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:22.526332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9782b4ae-a005-4186-b2e7-5dcdc75f985a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef3744018d1845a08a540f40db6b08b5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:22.526332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9782b4ae-a005-4186-b2e7-5dcdc75f985a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef3744018d1845a08a540f40db6b08b5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:22.544835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-9782b4ae-a005-4186-b2e7-5dcdc75f985a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef3744018d1845a08a540f40db6b08b5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:22.571738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1211/2413] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:21 2022] DELETE /identity/v3/users/ef3744018d1845a08a540f40db6b08b5 => generated 0 bytes in 948 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:23.470036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:23.711127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853063e6-c2c6-4ff6-866c-615146e362d9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:23.712703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-853063e6-c2c6-4ff6-866c-615146e362d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:23.744421 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853063e6-c2c6-4ff6-866c-615146e362d9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:23.877906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4956b28-be10-4602-8c82-6ca5fb4597c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:23.878984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4956b28-be10-4602-8c82-6ca5fb4597c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:23.879590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4956b28-be10-4602-8c82-6ca5fb4597c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:23.880097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4956b28-be10-4602-8c82-6ca5fb4597c6 admin admin] PATH_INFO: `/v3/projects/b44098cd716440028d9795ed785be3bb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:23.881719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4956b28-be10-4602-8c82-6ca5fb4597c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b44098cd716440028d9795ed785be3bb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:23.883982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4956b28-be10-4602-8c82-6ca5fb4597c6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:24.108468 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b4956b28-be10-4602-8c82-6ca5fb4597c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b44098cd716440028d9795ed785be3bb', 'request_id': 'req-b4956b28-be10-4602-8c82-6ca5fb4597c6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:24.123279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b4956b28-be10-4602-8c82-6ca5fb4597c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b44098cd716440028d9795ed785be3bb', 'request_id': 'req-b4956b28-be10-4602-8c82-6ca5fb4597c6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:24.287274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b4956b28-be10-4602-8c82-6ca5fb4597c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b44098cd716440028d9795ed785be3bb', 'request_id': 'req-b4956b28-be10-4602-8c82-6ca5fb4597c6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:24.325600 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b4956b28-be10-4602-8c82-6ca5fb4597c6 admin admin] The token cache is being invalidate because project b44098cd716440028d9795ed785be3bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:24.326663 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b4956b28-be10-4602-8c82-6ca5fb4597c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:24.333362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b4956b28-be10-4602-8c82-6ca5fb4597c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:24.336658 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1203/2414] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:23 2022] DELETE /identity/v3/projects/b44098cd716440028d9795ed785be3bb => generated 0 bytes in 626 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:24.345768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07649ee5-324e-4782-9235-424e77790f83 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:24.346574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07649ee5-324e-4782-9235-424e77790f83 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:24.346574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07649ee5-324e-4782-9235-424e77790f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:24.375677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-07649ee5-324e-4782-9235-424e77790f83 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:24.378300 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-07649ee5-324e-4782-9235-424e77790f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:24.524168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1212/2415] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:24.529833 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91f747-428c-480a-86ed-57e4b554ace2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:24.530115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91f747-428c-480a-86ed-57e4b554ace2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:24.530390 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91f747-428c-480a-86ed-57e4b554ace2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:24.645533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ff91f747-428c-480a-86ed-57e4b554ace2 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:24.647830 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ff91f747-428c-480a-86ed-57e4b554ace2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:24.694551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1204/2416] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:24.699545 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07649ee5-324e-4782-9235-424e77790f83 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:24.745980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07649ee5-324e-4782-9235-424e77790f83 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:24.798319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b823f7-d29a-4668-877f-80bad5582037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:24.799082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b823f7-d29a-4668-877f-80bad5582037 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:24.799533 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b823f7-d29a-4668-877f-80bad5582037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:24.800147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b823f7-d29a-4668-877f-80bad5582037 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:24.801462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b823f7-d29a-4668-877f-80bad5582037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:06:24.802096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b823f7-d29a-4668-877f-80bad5582037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:24.810572 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b823f7-d29a-4668-877f-80bad5582037 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:24.849186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1213/2417] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:06:24.863843 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91f747-428c-480a-86ed-57e4b554ace2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:24.939453 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91f747-428c-480a-86ed-57e4b554ace2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:24.986751 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77f68d-c14f-454b-b943-bb69894bcfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:24.995130 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77f68d-c14f-454b-b943-bb69894bcfb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:24.996284 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77f68d-c14f-454b-b943-bb69894bcfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:25.001693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77f68d-c14f-454b-b943-bb69894bcfb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:25.014881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77f68d-c14f-454b-b943-bb69894bcfb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:25.023214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77f68d-c14f-454b-b943-bb69894bcfb9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:25.217362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1205/2418] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:24 2022] POST /identity/v3/projects => generated 383 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:25.224234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b823f7-d29a-4668-877f-80bad5582037 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:25.237813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b823f7-d29a-4668-877f-80bad5582037 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:25.259386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:25.455996 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c78962-1dc9-4b6c-8776-911ecbba2655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:25.458402 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78962-1dc9-4b6c-8776-911ecbba2655 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:25.471888 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78962-1dc9-4b6c-8776-911ecbba2655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:25.475409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78962-1dc9-4b6c-8776-911ecbba2655 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:25.476952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c78962-1dc9-4b6c-8776-911ecbba2655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:25.487463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c78962-1dc9-4b6c-8776-911ecbba2655 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:25.610692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1214/2419] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:25 2022] POST /identity/v3/users => generated 292 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:06:25.617188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77f68d-c14f-454b-b943-bb69894bcfb9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:25.680276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:25.707147 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77f68d-c14f-454b-b943-bb69894bcfb9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:25.750973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf13e39-69f5-45c5-9a04-b1e920d6ad20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:25.754964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf13e39-69f5-45c5-9a04-b1e920d6ad20 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:25.760960 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf13e39-69f5-45c5-9a04-b1e920d6ad20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:25.763974 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf13e39-69f5-45c5-9a04-b1e920d6ad20 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:25.765638 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf13e39-69f5-45c5-9a04-b1e920d6ad20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:25.780402 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf13e39-69f5-45c5-9a04-b1e920d6ad20 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:25.820925 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1206/2420] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:25.826282 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c78962-1dc9-4b6c-8776-911ecbba2655 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:25.860489 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c78962-1dc9-4b6c-8776-911ecbba2655 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:25.867415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6fcfa-57e9-44ae-ab94-074b6a01a8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:25.868089 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6fcfa-57e9-44ae-ab94-074b6a01a8a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:25.870713 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6fcfa-57e9-44ae-ab94-074b6a01a8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:25.871252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6fcfa-57e9-44ae-ab94-074b6a01a8a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:25.872126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6fcfa-57e9-44ae-ab94-074b6a01a8a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:25.873970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6fcfa-57e9-44ae-ab94-074b6a01a8a4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:25.893443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1215/2421] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:25.908980 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf13e39-69f5-45c5-9a04-b1e920d6ad20 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:25.941140 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf13e39-69f5-45c5-9a04-b1e920d6ad20 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:25.959099 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6561a-03e4-4644-80be-ffb620e75d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:25.960183 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6561a-03e4-4644-80be-ffb620e75d0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:25.960849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6561a-03e4-4644-80be-ffb620e75d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:25.961522 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6561a-03e4-4644-80be-ffb620e75d0c admin admin] PATH_INFO: `/v3/projects/ac5c1a95f2cc4462a74d32d4aef7c50e/users/e18c8c8837204d04b9b95b91ed058cc7/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:25.997668 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6561a-03e4-4644-80be-ffb620e75d0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac5c1a95f2cc4462a74d32d4aef7c50e, user_id=e18c8c8837204d04b9b95b91ed058cc7, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:26.005740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba6561a-03e4-4644-80be-ffb620e75d0c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:26.126199 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1207/2422] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:25 2022] PUT /identity/v3/projects/ac5c1a95f2cc4462a74d32d4aef7c50e/users/e18c8c8837204d04b9b95b91ed058cc7/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:26.147762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdb910-a88f-42e4-8615-954f76c86ca1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:26.148239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdb910-a88f-42e4-8615-954f76c86ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:26.148690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdb910-a88f-42e4-8615-954f76c86ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:26.208129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-eafdb910-a88f-42e4-8615-954f76c86ca1 None None] MFA Rules not processed for user `e18c8c8837204d04b9b95b91ed058cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:26.210677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-eafdb910-a88f-42e4-8615-954f76c86ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:26.336118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1216/2423] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:26.339447 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6561a-03e4-4644-80be-ffb620e75d0c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:26.558217 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6561a-03e4-4644-80be-ffb620e75d0c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:26.567018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4be4f-51e0-4bbe-9ea4-7330794c7e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:26.568148 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4be4f-51e0-4bbe-9ea4-7330794c7e15 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:26.568661 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4be4f-51e0-4bbe-9ea4-7330794c7e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:26.569093 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4be4f-51e0-4bbe-9ea4-7330794c7e15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:26.569976 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4be4f-51e0-4bbe-9ea4-7330794c7e15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:26.573152 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4be4f-51e0-4bbe-9ea4-7330794c7e15 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:26.893953 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1208/2424] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:26 2022] POST /identity/v3/projects => generated 385 bytes in 555 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:06:26.894884 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdb910-a88f-42e4-8615-954f76c86ca1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:26.921095 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdb910-a88f-42e4-8615-954f76c86ca1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:27.079366 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3966823-2d0d-41cb-8d2f-2aa401409c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:27.080935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3966823-2d0d-41cb-8d2f-2aa401409c57 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.081463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3966823-2d0d-41cb-8d2f-2aa401409c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.082039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3966823-2d0d-41cb-8d2f-2aa401409c57 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.083166 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3966823-2d0d-41cb-8d2f-2aa401409c57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:27.085583 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3966823-2d0d-41cb-8d2f-2aa401409c57 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:27.216054 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4be4f-51e0-4bbe-9ea4-7330794c7e15 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:27.216904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1217/2425] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:26 2022] POST /identity/v3/users => generated 293 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:27.234496 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4be4f-51e0-4bbe-9ea4-7330794c7e15 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:27.245364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28294a-630b-4649-a145-849fef2d17a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:27.246526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28294a-630b-4649-a145-849fef2d17a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.247034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28294a-630b-4649-a145-849fef2d17a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.247536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b28294a-630b-4649-a145-849fef2d17a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.248867 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28294a-630b-4649-a145-849fef2d17a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:27.253035 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b28294a-630b-4649-a145-849fef2d17a3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:27.276869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1209/2426] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:27.283529 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3966823-2d0d-41cb-8d2f-2aa401409c57 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:27.303440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3966823-2d0d-41cb-8d2f-2aa401409c57 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:27.320615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01926ef1-ca85-4244-b613-0724af7026ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:27.320615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01926ef1-ca85-4244-b613-0724af7026ed admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.320615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01926ef1-ca85-4244-b613-0724af7026ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.320615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01926ef1-ca85-4244-b613-0724af7026ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.321345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01926ef1-ca85-4244-b613-0724af7026ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:27.321345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01926ef1-ca85-4244-b613-0724af7026ed admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:27.345847 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1218/2427] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:27.349866 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28294a-630b-4649-a145-849fef2d17a3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:27.361438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b28294a-630b-4649-a145-849fef2d17a3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:27.367748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc10e52-b36f-4b5f-bbf2-0e541fdd7134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:27.369795 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc10e52-b36f-4b5f-bbf2-0e541fdd7134 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.370714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc10e52-b36f-4b5f-bbf2-0e541fdd7134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.371390 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc10e52-b36f-4b5f-bbf2-0e541fdd7134 admin admin] PATH_INFO: `/v3/projects/fdeeaf808629459bae7923e97ddceb57/users/b37abbbb272c4eb6a7b6f7079af04d29/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.405726 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc10e52-b36f-4b5f-bbf2-0e541fdd7134 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdeeaf808629459bae7923e97ddceb57, user_id=b37abbbb272c4eb6a7b6f7079af04d29, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:27.410839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc10e52-b36f-4b5f-bbf2-0e541fdd7134 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:27.521722 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1210/2428] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:27 2022] PUT /identity/v3/projects/fdeeaf808629459bae7923e97ddceb57/users/b37abbbb272c4eb6a7b6f7079af04d29/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:27.533473 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5ff12-7140-42a5-8202-aabf3c7c3c7d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.534698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5ff12-7140-42a5-8202-aabf3c7c3c7d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.535441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5ff12-7140-42a5-8202-aabf3c7c3c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.586225 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a3c5ff12-7140-42a5-8202-aabf3c7c3c7d None None] MFA Rules not processed for user `b37abbbb272c4eb6a7b6f7079af04d29`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:27.588069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a3c5ff12-7140-42a5-8202-aabf3c7c3c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:27.728362 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1219/2429] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:27 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:27.734063 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc10e52-b36f-4b5f-bbf2-0e541fdd7134 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:27.886195 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc10e52-b36f-4b5f-bbf2-0e541fdd7134 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:27.895357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e315f-a1fa-4d55-9fe0-494d15ed5e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:27.896538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e315f-a1fa-4d55-9fe0-494d15ed5e88 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:27.897089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e315f-a1fa-4d55-9fe0-494d15ed5e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:27.897692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e315f-a1fa-4d55-9fe0-494d15ed5e88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:27.898515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e315f-a1fa-4d55-9fe0-494d15ed5e88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:27.900890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e315f-a1fa-4d55-9fe0-494d15ed5e88 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:28.152889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1211/2430] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:27 2022] POST /identity/v3/projects => generated 383 bytes in 419 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:28.157637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5ff12-7140-42a5-8202-aabf3c7c3c7d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:28.178350 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5ff12-7140-42a5-8202-aabf3c7c3c7d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:28.292647 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491eedee-1e1b-4b3c-a00d-f568f2d11a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:28.294188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491eedee-1e1b-4b3c-a00d-f568f2d11a68 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:28.294720 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491eedee-1e1b-4b3c-a00d-f568f2d11a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:28.295190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491eedee-1e1b-4b3c-a00d-f568f2d11a68 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:28.296340 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491eedee-1e1b-4b3c-a00d-f568f2d11a68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:28.301034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491eedee-1e1b-4b3c-a00d-f568f2d11a68 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:28.383657 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:28.471165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1220/2431] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:28 2022] POST /identity/v3/users => generated 298 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:28.477071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e315f-a1fa-4d55-9fe0-494d15ed5e88 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:28.497997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e315f-a1fa-4d55-9fe0-494d15ed5e88 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:28.510647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bc90c-9b56-48fd-a124-03a61c98b575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:28.511388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bc90c-9b56-48fd-a124-03a61c98b575 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:28.511810 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bc90c-9b56-48fd-a124-03a61c98b575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:28.512957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522bc90c-9b56-48fd-a124-03a61c98b575 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:28.512957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522bc90c-9b56-48fd-a124-03a61c98b575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:28.515040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522bc90c-9b56-48fd-a124-03a61c98b575 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:28.527624 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1212/2432] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:28.533398 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491eedee-1e1b-4b3c-a00d-f568f2d11a68 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:28.544928 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491eedee-1e1b-4b3c-a00d-f568f2d11a68 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:28.551528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0c8ff-a1e0-4649-9641-cf85dbe27ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:28.552664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0c8ff-a1e0-4649-9641-cf85dbe27ccc admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:28.552664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0c8ff-a1e0-4649-9641-cf85dbe27ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:28.552990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0c8ff-a1e0-4649-9641-cf85dbe27ccc admin admin] PATH_INFO: `/v3/projects/455fc9e4f0234ff7bf8834e9dcc5392c/users/12db3405aec844a3a6af5daa62fd4e06/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:28.571480 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0c8ff-a1e0-4649-9641-cf85dbe27ccc admin admin] RBAC: Authorizing `identity:create_grant(project_id=455fc9e4f0234ff7bf8834e9dcc5392c, user_id=12db3405aec844a3a6af5daa62fd4e06, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:28.574473 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0c8ff-a1e0-4649-9641-cf85dbe27ccc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:28.780090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1221/2433] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:28 2022] PUT /identity/v3/projects/455fc9e4f0234ff7bf8834e9dcc5392c/users/12db3405aec844a3a6af5daa62fd4e06/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:28.785814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bc90c-9b56-48fd-a124-03a61c98b575 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:28.808420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522bc90c-9b56-48fd-a124-03a61c98b575 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:28.954789 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218769cf-e299-4a0a-b12d-46728fccc2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:28.955987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218769cf-e299-4a0a-b12d-46728fccc2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:28.956503 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218769cf-e299-4a0a-b12d-46728fccc2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:28.956982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218769cf-e299-4a0a-b12d-46728fccc2a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:28.958378 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218769cf-e299-4a0a-b12d-46728fccc2a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:28.962035 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218769cf-e299-4a0a-b12d-46728fccc2a4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:29.006595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1213/2434] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:28 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:29.012685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0c8ff-a1e0-4649-9641-cf85dbe27ccc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:29.036905 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0c8ff-a1e0-4649-9641-cf85dbe27ccc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:29.045552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c914d0-ea25-427c-9a83-82047bd12f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:29.048759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c914d0-ea25-427c-9a83-82047bd12f40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:29.049266 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c914d0-ea25-427c-9a83-82047bd12f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:29.049769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c914d0-ea25-427c-9a83-82047bd12f40 admin admin] PATH_INFO: `/v3/projects/455fc9e4f0234ff7bf8834e9dcc5392c/users/12db3405aec844a3a6af5daa62fd4e06/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:29.058354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c914d0-ea25-427c-9a83-82047bd12f40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=455fc9e4f0234ff7bf8834e9dcc5392c, user_id=12db3405aec844a3a6af5daa62fd4e06, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:29.063814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c914d0-ea25-427c-9a83-82047bd12f40 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:29.088156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1222/2435] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:29 2022] PUT /identity/v3/projects/455fc9e4f0234ff7bf8834e9dcc5392c/users/12db3405aec844a3a6af5daa62fd4e06/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:29.106390 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488f2cf-7b99-4b3a-ba22-197f02100300 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:29.106863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488f2cf-7b99-4b3a-ba22-197f02100300 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:29.107209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0488f2cf-7b99-4b3a-ba22-197f02100300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:29.180745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0488f2cf-7b99-4b3a-ba22-197f02100300 None None] MFA Rules not processed for user `12db3405aec844a3a6af5daa62fd4e06`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:29.183376 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0488f2cf-7b99-4b3a-ba22-197f02100300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:29.400570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1214/2436] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:29.420685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c914d0-ea25-427c-9a83-82047bd12f40 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:29.424121 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d1c914d0-ea25-427c-9a83-82047bd12f40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:29.601167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c914d0-ea25-427c-9a83-82047bd12f40 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:29.608262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a363f5f-2c37-42c9-aeba-7a0bb2f51c34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:29.609108 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a363f5f-2c37-42c9-aeba-7a0bb2f51c34 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:29.610106 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a363f5f-2c37-42c9-aeba-7a0bb2f51c34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:29.610518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a363f5f-2c37-42c9-aeba-7a0bb2f51c34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:29.718391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:29.796222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a363f5f-2c37-42c9-aeba-7a0bb2f51c34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:29.799381 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a363f5f-2c37-42c9-aeba-7a0bb2f51c34 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:29.831696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1223/2437] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:30.103369 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:31.004134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d3fdc-ce04-4b98-b717-745372ff41c2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:31.013570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d3fdc-ce04-4b98-b717-745372ff41c2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:31.013570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d3fdc-ce04-4b98-b717-745372ff41c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:31.051954 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-769d3fdc-ce04-4b98-b717-745372ff41c2 None None] MFA Rules not processed for user `183fe84502664f49a701a5e9eb4f6cd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:31.053225 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-769d3fdc-ce04-4b98-b717-745372ff41c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:31.174652 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:31.259622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1215/2438] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:31.264879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a363f5f-2c37-42c9-aeba-7a0bb2f51c34 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:31.268171 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4a363f5f-2c37-42c9-aeba-7a0bb2f51c34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:31.374143 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:31.485200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a363f5f-2c37-42c9-aeba-7a0bb2f51c34 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:31.494897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ddd07-6218-4a70-861f-e2184633bf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:31.496066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ddd07-6218-4a70-861f-e2184633bf9f admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:31.496596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ddd07-6218-4a70-861f-e2184633bf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:31.497374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3ddd07-6218-4a70-861f-e2184633bf9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:31.498952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ddd07-6218-4a70-861f-e2184633bf9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:31.504413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3ddd07-6218-4a70-861f-e2184633bf9f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:31.684630 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1224/2439] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:31 2022] POST /identity/v3/projects => generated 371 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:31.685056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d3fdc-ce04-4b98-b717-745372ff41c2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:31.699973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d3fdc-ce04-4b98-b717-745372ff41c2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:31.784282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d92b19-1165-451c-98c6-a16a5c2a9e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:31.785024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d92b19-1165-451c-98c6-a16a5c2a9e17 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:31.785379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d92b19-1165-451c-98c6-a16a5c2a9e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:31.785774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d92b19-1165-451c-98c6-a16a5c2a9e17 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:31.786498 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d92b19-1165-451c-98c6-a16a5c2a9e17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:31.788729 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d92b19-1165-451c-98c6-a16a5c2a9e17 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:31.901891 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ddd07-6218-4a70-861f-e2184633bf9f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:31.909804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1216/2440] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:31 2022] POST /identity/v3/users => generated 292 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:31.916157 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3ddd07-6218-4a70-861f-e2184633bf9f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:31.923318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c388ed9-3c91-4b8c-b999-8e1e43cf0ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:31.924119 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c388ed9-3c91-4b8c-b999-8e1e43cf0ada admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:31.924476 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c388ed9-3c91-4b8c-b999-8e1e43cf0ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:31.925037 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c388ed9-3c91-4b8c-b999-8e1e43cf0ada admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:31.925814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c388ed9-3c91-4b8c-b999-8e1e43cf0ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:31.927543 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c388ed9-3c91-4b8c-b999-8e1e43cf0ada admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:31.947747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1225/2441] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:06:31.953328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d92b19-1165-451c-98c6-a16a5c2a9e17 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:31.980071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d92b19-1165-451c-98c6-a16a5c2a9e17 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:31.999158 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c2d50-ec4e-453f-bd5b-3d76f95ac5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:32.000587 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c2d50-ec4e-453f-bd5b-3d76f95ac5e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:32.000955 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c2d50-ec4e-453f-bd5b-3d76f95ac5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:32.001301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07c2d50-ec4e-453f-bd5b-3d76f95ac5e5 admin admin] PATH_INFO: `/v3/projects/35275742c4a04744be7a7ef07a7f9356/users/62da2adabc3c4ab6959b1b83ea903e38/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:32.031407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c2d50-ec4e-453f-bd5b-3d76f95ac5e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35275742c4a04744be7a7ef07a7f9356, user_id=62da2adabc3c4ab6959b1b83ea903e38, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:32.037282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07c2d50-ec4e-453f-bd5b-3d76f95ac5e5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:32.140645 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1217/2442] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:31 2022] PUT /identity/v3/projects/35275742c4a04744be7a7ef07a7f9356/users/62da2adabc3c4ab6959b1b83ea903e38/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:32.145148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c388ed9-3c91-4b8c-b999-8e1e43cf0ada admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:32.158047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c388ed9-3c91-4b8c-b999-8e1e43cf0ada admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:32.244643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e25d1-f5c0-4575-b406-c0969916028b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:32.245755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e25d1-f5c0-4575-b406-c0969916028b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:32.245755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e25d1-f5c0-4575-b406-c0969916028b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:32.246211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e25d1-f5c0-4575-b406-c0969916028b admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:32.247293 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e25d1-f5c0-4575-b406-c0969916028b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:32.250076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e25d1-f5c0-4575-b406-c0969916028b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:32.263168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1226/2443] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:32 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:32.268558 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c2d50-ec4e-453f-bd5b-3d76f95ac5e5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:32.285756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07c2d50-ec4e-453f-bd5b-3d76f95ac5e5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:32.292172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a868b97-8011-422c-972d-b1e14f3e1bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:32.293374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a868b97-8011-422c-972d-b1e14f3e1bce admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:32.293858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a868b97-8011-422c-972d-b1e14f3e1bce admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:32.294333 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a868b97-8011-422c-972d-b1e14f3e1bce admin admin] PATH_INFO: `/v3/projects/35275742c4a04744be7a7ef07a7f9356/users/62da2adabc3c4ab6959b1b83ea903e38/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:32.297910 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a868b97-8011-422c-972d-b1e14f3e1bce admin admin] RBAC: Authorizing `identity:create_grant(project_id=35275742c4a04744be7a7ef07a7f9356, user_id=62da2adabc3c4ab6959b1b83ea903e38, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:32.300807 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a868b97-8011-422c-972d-b1e14f3e1bce admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:32.324143 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1218/2444] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:32 2022] PUT /identity/v3/projects/35275742c4a04744be7a7ef07a7f9356/users/62da2adabc3c4ab6959b1b83ea903e38/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:32.775304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:33.549487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:36.083212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:36.168313 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e25d1-f5c0-4575-b406-c0969916028b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:36.292487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e25d1-f5c0-4575-b406-c0969916028b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:36.404131 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24946e89-e148-40a9-81e6-e4a4f6c51ef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:36.409381 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24946e89-e148-40a9-81e6-e4a4f6c51ef5 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:36.409668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24946e89-e148-40a9-81e6-e4a4f6c51ef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:36.412158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24946e89-e148-40a9-81e6-e4a4f6c51ef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:36.570607 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24946e89-e148-40a9-81e6-e4a4f6c51ef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:36.575232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24946e89-e148-40a9-81e6-e4a4f6c51ef5 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:36.593479 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1227/2445] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:36.625523 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a868b97-8011-422c-972d-b1e14f3e1bce admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:36.628714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0a868b97-8011-422c-972d-b1e14f3e1bce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:36.768011 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a868b97-8011-422c-972d-b1e14f3e1bce admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:36.780039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdd48b-0c1b-4c24-b23d-895bf3f774d6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:36.797957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdd48b-0c1b-4c24-b23d-895bf3f774d6 service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:36.799748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdd48b-0c1b-4c24-b23d-895bf3f774d6 service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:36.799882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffdd48b-0c1b-4c24-b23d-895bf3f774d6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:36.811299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3ffdd48b-0c1b-4c24-b23d-895bf3f774d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:36.833483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffdd48b-0c1b-4c24-b23d-895bf3f774d6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:36.836443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffdd48b-0c1b-4c24-b23d-895bf3f774d6 service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:36.852057 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1219/2446] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:37.157683 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:38.208357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:40.070794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=8,keystone:SELECT=87,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:40.111388 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f8a04-80af-4289-be78-58dcf6e2f429 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:40.111940 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f8a04-80af-4289-be78-58dcf6e2f429 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:40.112409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46f8a04-80af-4289-be78-58dcf6e2f429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:40.132626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdd48b-0c1b-4c24-b23d-895bf3f774d6 service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:40.134093 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3ffdd48b-0c1b-4c24-b23d-895bf3f774d6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:40.152200 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffdd48b-0c1b-4c24-b23d-895bf3f774d6 service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:40.153340 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b46f8a04-80af-4289-be78-58dcf6e2f429 None None] MFA Rules not processed for user `62da2adabc3c4ab6959b1b83ea903e38`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:40.155984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b46f8a04-80af-4289-be78-58dcf6e2f429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:40.158609 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911256a6-3688-449a-8d41-828b78f9afa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:40.159246 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911256a6-3688-449a-8d41-828b78f9afa5 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:40.159527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911256a6-3688-449a-8d41-828b78f9afa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:40.159836 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911256a6-3688-449a-8d41-828b78f9afa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:40.247457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911256a6-3688-449a-8d41-828b78f9afa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:40.249229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911256a6-3688-449a-8d41-828b78f9afa5 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:40.256957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1228/2447] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:40 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:40.278927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f8a04-80af-4289-be78-58dcf6e2f429 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:40.279172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1220/2448] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:06:40.279317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b46f8a04-80af-4289-be78-58dcf6e2f429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:40.402939 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46f8a04-80af-4289-be78-58dcf6e2f429 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:40.411487 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70277a-705f-4fef-894c-625ce2a16147 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:40.412144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70277a-705f-4fef-894c-625ce2a16147 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:40.412427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70277a-705f-4fef-894c-625ce2a16147 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:40.412706 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70277a-705f-4fef-894c-625ce2a16147 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:40.482374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70277a-705f-4fef-894c-625ce2a16147 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:40.484711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70277a-705f-4fef-894c-625ce2a16147 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:40.511873 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1229/2449] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:06:40.989913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:43.092943 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:43.438781 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:45.202394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70890a2f-53e7-4e8d-b842-1f166c952e0f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:45.202394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70890a2f-53e7-4e8d-b842-1f166c952e0f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:45.202394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70890a2f-53e7-4e8d-b842-1f166c952e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:45.228663 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-70890a2f-53e7-4e8d-b842-1f166c952e0f None None] MFA Rules not processed for user `183fe84502664f49a701a5e9eb4f6cd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:45.232622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-70890a2f-53e7-4e8d-b842-1f166c952e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:45.246586 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1221/2450] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:46.425519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:46.867211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=80,keystone:INSERT=5,keystone:DELETE=15 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:47.950707 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:48.322129 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:48.623898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:49.111737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:49.397223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:50.612616 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:50.869522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:51.921544 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4e052-336c-409d-9145-89d5ced69fff None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:51.922220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4e052-336c-409d-9145-89d5ced69fff None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:51.922963 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa4e052-336c-409d-9145-89d5ced69fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:51.946349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-daa4e052-336c-409d-9145-89d5ced69fff None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:51.947839 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-daa4e052-336c-409d-9145-89d5ced69fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:52.021748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70890a2f-53e7-4e8d-b842-1f166c952e0f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:52.039482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1230/2451] 10.222.0.67 () {58 vars in 1009 bytes} [Mon Sep 26 18:06:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:52.065014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4e052-336c-409d-9145-89d5ced69fff None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:52.066789 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-daa4e052-336c-409d-9145-89d5ced69fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:52.121020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70890a2f-53e7-4e8d-b842-1f166c952e0f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:52.127226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5951af-c9ed-48cb-819f-587e8b8daade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:52.127920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5951af-c9ed-48cb-819f-587e8b8daade admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:52.128199 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5951af-c9ed-48cb-819f-587e8b8daade admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:52.128469 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5951af-c9ed-48cb-819f-587e8b8daade admin admin] PATH_INFO: `/v3/users/e18c8c8837204d04b9b95b91ed058cc7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:52.129665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5951af-c9ed-48cb-819f-587e8b8daade admin admin] RBAC: Authorizing `identity:delete_user(user_id=e18c8c8837204d04b9b95b91ed058cc7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:52.131612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5951af-c9ed-48cb-819f-587e8b8daade admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:52.167257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa4e052-336c-409d-9145-89d5ced69fff None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:52.167257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc326a4a-8fe3-4c82-a2c8-ed361ee99918 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:52.167257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc326a4a-8fe3-4c82-a2c8-ed361ee99918 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:52.167549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc326a4a-8fe3-4c82-a2c8-ed361ee99918 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:52.167711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc326a4a-8fe3-4c82-a2c8-ed361ee99918 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:52.177823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc326a4a-8fe3-4c82-a2c8-ed361ee99918 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:52.179113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc326a4a-8fe3-4c82-a2c8-ed361ee99918 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:52.188618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1231/2452] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:06:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:06:52.190355 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc326a4a-8fe3-4c82-a2c8-ed361ee99918 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:52.190958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cc326a4a-8fe3-4c82-a2c8-ed361ee99918 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:52.205232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc326a4a-8fe3-4c82-a2c8-ed361ee99918 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:52.210190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b7e7f-c59d-45d1-95b0-11464a97172a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:52.211609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b7e7f-c59d-45d1-95b0-11464a97172a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:52.211939 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b7e7f-c59d-45d1-95b0-11464a97172a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:52.212262 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b7e7f-c59d-45d1-95b0-11464a97172a admin admin] PATH_INFO: `/v3/users/0f7418db7d8948ec8a165793fd99c406` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:52.213461 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b7e7f-c59d-45d1-95b0-11464a97172a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f7418db7d8948ec8a165793fd99c406)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:52.215365 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672b7e7f-c59d-45d1-95b0-11464a97172a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:52.659512 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8d5951af-c9ed-48cb-819f-587e8b8daade admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e18c8c8837204d04b9b95b91ed058cc7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.795688 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-672b7e7f-c59d-45d1-95b0-11464a97172a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f7418db7d8948ec8a165793fd99c406'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.907607 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8d5951af-c9ed-48cb-819f-587e8b8daade admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e18c8c8837204d04b9b95b91ed058cc7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.908150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8d5951af-c9ed-48cb-819f-587e8b8daade admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e18c8c8837204d04b9b95b91ed058cc7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.909380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8d5951af-c9ed-48cb-819f-587e8b8daade admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e18c8c8837204d04b9b95b91ed058cc7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.918972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8d5951af-c9ed-48cb-819f-587e8b8daade admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e18c8c8837204d04b9b95b91ed058cc7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:52.942331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1222/2453] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:52 2022] DELETE /identity/v3/users/e18c8c8837204d04b9b95b91ed058cc7 => generated 0 bytes in 921 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:52.947712 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113f00c-37fb-44c9-91ae-3651a9c79f61 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:52.948001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113f00c-37fb-44c9-91ae-3651a9c79f61 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:52.948277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113f00c-37fb-44c9-91ae-3651a9c79f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:52.970824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5113f00c-37fb-44c9-91ae-3651a9c79f61 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:52.973145 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5113f00c-37fb-44c9-91ae-3651a9c79f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:53.059254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1223/2454] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:53.065817 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-672b7e7f-c59d-45d1-95b0-11464a97172a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f7418db7d8948ec8a165793fd99c406'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.069036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-672b7e7f-c59d-45d1-95b0-11464a97172a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f7418db7d8948ec8a165793fd99c406'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.070495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-672b7e7f-c59d-45d1-95b0-11464a97172a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f7418db7d8948ec8a165793fd99c406'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.088786 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-672b7e7f-c59d-45d1-95b0-11464a97172a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f7418db7d8948ec8a165793fd99c406'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:53.092904 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113f00c-37fb-44c9-91ae-3651a9c79f61 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:53.095806 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5113f00c-37fb-44c9-91ae-3651a9c79f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:53.110669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:53.124500 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1232/2455] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:52 2022] DELETE /identity/v3/users/0f7418db7d8948ec8a165793fd99c406 => generated 0 bytes in 935 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:53.130070 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c257b8-bfab-4682-9043-55394fdb5b1f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:53.130435 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c257b8-bfab-4682-9043-55394fdb5b1f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:53.130773 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c257b8-bfab-4682-9043-55394fdb5b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:53.185968 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-35c257b8-bfab-4682-9043-55394fdb5b1f None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:53.187704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-35c257b8-bfab-4682-9043-55394fdb5b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:53.220911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113f00c-37fb-44c9-91ae-3651a9c79f61 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:53.287690 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99e70f-3be9-4999-ba61-2230750764aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:53.288395 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99e70f-3be9-4999-ba61-2230750764aa service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:53.288677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99e70f-3be9-4999-ba61-2230750764aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:53.288950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99e70f-3be9-4999-ba61-2230750764aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:53.291436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5a99e70f-3be9-4999-ba61-2230750764aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:53.295102 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1233/2456] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:06:53.302480 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c257b8-bfab-4682-9043-55394fdb5b1f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:53.304795 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-35c257b8-bfab-4682-9043-55394fdb5b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:53.316425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99e70f-3be9-4999-ba61-2230750764aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:53.317954 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99e70f-3be9-4999-ba61-2230750764aa service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:53.328295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c257b8-bfab-4682-9043-55394fdb5b1f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:53.328902 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1224/2457] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:06:53.334761 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194401b5-5d1e-4619-8fc9-d0c5863b2720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:53.335882 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194401b5-5d1e-4619-8fc9-d0c5863b2720 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:53.336389 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194401b5-5d1e-4619-8fc9-d0c5863b2720 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:53.336880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194401b5-5d1e-4619-8fc9-d0c5863b2720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:53.357307 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194401b5-5d1e-4619-8fc9-d0c5863b2720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:53.360249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194401b5-5d1e-4619-8fc9-d0c5863b2720 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:53.380399 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1234/2458] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:06:54.032597 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99e70f-3be9-4999-ba61-2230750764aa service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:54.036070 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5a99e70f-3be9-4999-ba61-2230750764aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:54.037848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:54.053776 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194401b5-5d1e-4619-8fc9-d0c5863b2720 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:54.056702 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-194401b5-5d1e-4619-8fc9-d0c5863b2720 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:54.076009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99e70f-3be9-4999-ba61-2230750764aa service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:54.143306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f2507-04de-465e-b391-abafa993a060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:54.144026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f2507-04de-465e-b391-abafa993a060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:54.144442 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f2507-04de-465e-b391-abafa993a060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:54.155842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7f2507-04de-465e-b391-abafa993a060 admin admin] PATH_INFO: `/v3/projects/ac5c1a95f2cc4462a74d32d4aef7c50e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:54.156158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194401b5-5d1e-4619-8fc9-d0c5863b2720 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:54.158069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f2507-04de-465e-b391-abafa993a060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac5c1a95f2cc4462a74d32d4aef7c50e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:54.160488 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7f2507-04de-465e-b391-abafa993a060 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:54.165746 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6d627-d4a3-4300-8751-b49a4c6caf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:54.166473 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6d627-d4a3-4300-8751-b49a4c6caf85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:54.166649 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6d627-d4a3-4300-8751-b49a4c6caf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:54.167423 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6d627-d4a3-4300-8751-b49a4c6caf85 admin admin] PATH_INFO: `/v3/projects/b62230b4cd3943c5bf03d271d83af6a5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:54.168905 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6d627-d4a3-4300-8751-b49a4c6caf85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b62230b4cd3943c5bf03d271d83af6a5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:54.170922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6d627-d4a3-4300-8751-b49a4c6caf85 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:54.352593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b7f6d627-d4a3-4300-8751-b49a4c6caf85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b62230b4cd3943c5bf03d271d83af6a5', 'request_id': 'req-b7f6d627-d4a3-4300-8751-b49a4c6caf85'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:54.356986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f7f2507-04de-465e-b391-abafa993a060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac5c1a95f2cc4462a74d32d4aef7c50e', 'request_id': 'req-2f7f2507-04de-465e-b391-abafa993a060'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:54.366179 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b7f6d627-d4a3-4300-8751-b49a4c6caf85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b62230b4cd3943c5bf03d271d83af6a5', 'request_id': 'req-b7f6d627-d4a3-4300-8751-b49a4c6caf85'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:54.366921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f7f2507-04de-465e-b391-abafa993a060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac5c1a95f2cc4462a74d32d4aef7c50e', 'request_id': 'req-2f7f2507-04de-465e-b391-abafa993a060'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:54.494396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f7f2507-04de-465e-b391-abafa993a060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac5c1a95f2cc4462a74d32d4aef7c50e', 'request_id': 'req-2f7f2507-04de-465e-b391-abafa993a060'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:54.517729 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f7f2507-04de-465e-b391-abafa993a060 admin admin] The token cache is being invalidate because project ac5c1a95f2cc4462a74d32d4aef7c50e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:54.518091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f7f2507-04de-465e-b391-abafa993a060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:54.519561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f7f2507-04de-465e-b391-abafa993a060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:54.522201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1225/2459] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:54 2022] DELETE /identity/v3/projects/ac5c1a95f2cc4462a74d32d4aef7c50e => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:54.526923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f2507-04de-465e-b391-abafa993a060 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:54.529327 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2f7f2507-04de-465e-b391-abafa993a060 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:54.664309 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7f2507-04de-465e-b391-abafa993a060 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:54.671000 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028433f0-5fc2-44ed-a9be-adfff2272a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:54.672659 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028433f0-5fc2-44ed-a9be-adfff2272a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:54.673179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028433f0-5fc2-44ed-a9be-adfff2272a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:54.673693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028433f0-5fc2-44ed-a9be-adfff2272a5b admin admin] PATH_INFO: `/v3/users/b37abbbb272c4eb6a7b6f7079af04d29` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:54.675197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028433f0-5fc2-44ed-a9be-adfff2272a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b37abbbb272c4eb6a7b6f7079af04d29)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:54.677748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028433f0-5fc2-44ed-a9be-adfff2272a5b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:54.681572 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b7f6d627-d4a3-4300-8751-b49a4c6caf85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b62230b4cd3943c5bf03d271d83af6a5', 'request_id': 'req-b7f6d627-d4a3-4300-8751-b49a4c6caf85'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:54.705018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b7f6d627-d4a3-4300-8751-b49a4c6caf85 admin admin] The token cache is being invalidate because project b62230b4cd3943c5bf03d271d83af6a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:54.705018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b7f6d627-d4a3-4300-8751-b49a4c6caf85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:54.705018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b7f6d627-d4a3-4300-8751-b49a4c6caf85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:54.705754 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1235/2460] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:54 2022] DELETE /identity/v3/projects/b62230b4cd3943c5bf03d271d83af6a5 => generated 0 bytes in 652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:55.093202 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-028433f0-5fc2-44ed-a9be-adfff2272a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b37abbbb272c4eb6a7b6f7079af04d29'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:55.197921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-028433f0-5fc2-44ed-a9be-adfff2272a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b37abbbb272c4eb6a7b6f7079af04d29'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:55.199155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-028433f0-5fc2-44ed-a9be-adfff2272a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b37abbbb272c4eb6a7b6f7079af04d29'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:55.199957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-028433f0-5fc2-44ed-a9be-adfff2272a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b37abbbb272c4eb6a7b6f7079af04d29'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:55.213932 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-028433f0-5fc2-44ed-a9be-adfff2272a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b37abbbb272c4eb6a7b6f7079af04d29'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:55.241027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1226/2461] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:54 2022] DELETE /identity/v3/users/b37abbbb272c4eb6a7b6f7079af04d29 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:56.022194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6d627-d4a3-4300-8751-b49a4c6caf85 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.024723 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b7f6d627-d4a3-4300-8751-b49a4c6caf85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:56.113023 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:56.144347 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6d627-d4a3-4300-8751-b49a4c6caf85 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:56.225687 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899194bc-baf5-49e0-bb5b-702c83847257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.228812 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899194bc-baf5-49e0-bb5b-702c83847257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.228812 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899194bc-baf5-49e0-bb5b-702c83847257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.228812 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899194bc-baf5-49e0-bb5b-702c83847257 admin admin] PATH_INFO: `/v3/projects/fdeeaf808629459bae7923e97ddceb57` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.228812 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899194bc-baf5-49e0-bb5b-702c83847257 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdeeaf808629459bae7923e97ddceb57)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.230116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899194bc-baf5-49e0-bb5b-702c83847257 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:56.269354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:56.486465 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-899194bc-baf5-49e0-bb5b-702c83847257 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdeeaf808629459bae7923e97ddceb57', 'request_id': 'req-899194bc-baf5-49e0-bb5b-702c83847257'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:56.498287 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-899194bc-baf5-49e0-bb5b-702c83847257 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdeeaf808629459bae7923e97ddceb57', 'request_id': 'req-899194bc-baf5-49e0-bb5b-702c83847257'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:56.610017 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-899194bc-baf5-49e0-bb5b-702c83847257 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdeeaf808629459bae7923e97ddceb57', 'request_id': 'req-899194bc-baf5-49e0-bb5b-702c83847257'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:56.636518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-899194bc-baf5-49e0-bb5b-702c83847257 admin admin] The token cache is being invalidate because project fdeeaf808629459bae7923e97ddceb57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:56.636518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-899194bc-baf5-49e0-bb5b-702c83847257 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:56.637300 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-899194bc-baf5-49e0-bb5b-702c83847257 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:56.643910 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1236/2462] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:56 2022] DELETE /identity/v3/projects/fdeeaf808629459bae7923e97ddceb57 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:56.645049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028433f0-5fc2-44ed-a9be-adfff2272a5b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:56.647086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-028433f0-5fc2-44ed-a9be-adfff2272a5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:56.668115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028433f0-5fc2-44ed-a9be-adfff2272a5b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:56.768450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aa19d8-86a4-4f83-851d-ddda30443abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:56.769628 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa19d8-86a4-4f83-851d-ddda30443abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:56.770034 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa19d8-86a4-4f83-851d-ddda30443abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:56.770561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa19d8-86a4-4f83-851d-ddda30443abb admin admin] PATH_INFO: `/v3/users/12db3405aec844a3a6af5daa62fd4e06` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:56.772657 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa19d8-86a4-4f83-851d-ddda30443abb admin admin] RBAC: Authorizing `identity:delete_user(user_id=12db3405aec844a3a6af5daa62fd4e06)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:56.775365 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aa19d8-86a4-4f83-851d-ddda30443abb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:57.063256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-20aa19d8-86a4-4f83-851d-ddda30443abb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12db3405aec844a3a6af5daa62fd4e06'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:57.191532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-20aa19d8-86a4-4f83-851d-ddda30443abb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12db3405aec844a3a6af5daa62fd4e06'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:57.192595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-20aa19d8-86a4-4f83-851d-ddda30443abb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12db3405aec844a3a6af5daa62fd4e06'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:57.193563 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-20aa19d8-86a4-4f83-851d-ddda30443abb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12db3405aec844a3a6af5daa62fd4e06'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:57.208491 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-20aa19d8-86a4-4f83-851d-ddda30443abb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12db3405aec844a3a6af5daa62fd4e06'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:57.257370 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1227/2463] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:06:56 2022] DELETE /identity/v3/users/12db3405aec844a3a6af5daa62fd4e06 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:06:57.328285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:57.946273 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899194bc-baf5-49e0-bb5b-702c83847257 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:57.949213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-899194bc-baf5-49e0-bb5b-702c83847257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:57.992668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899194bc-baf5-49e0-bb5b-702c83847257 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:58.120926 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cbdce7-23d7-48ef-9d3c-136c1a046ca5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:58.121916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbdce7-23d7-48ef-9d3c-136c1a046ca5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:58.122361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbdce7-23d7-48ef-9d3c-136c1a046ca5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:58.122738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cbdce7-23d7-48ef-9d3c-136c1a046ca5 admin admin] PATH_INFO: `/v3/projects/455fc9e4f0234ff7bf8834e9dcc5392c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:58.125252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbdce7-23d7-48ef-9d3c-136c1a046ca5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=455fc9e4f0234ff7bf8834e9dcc5392c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:58.128872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cbdce7-23d7-48ef-9d3c-136c1a046ca5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:58.340671 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-25cbdce7-23d7-48ef-9d3c-136c1a046ca5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '455fc9e4f0234ff7bf8834e9dcc5392c', 'request_id': 'req-25cbdce7-23d7-48ef-9d3c-136c1a046ca5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:58.350633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-25cbdce7-23d7-48ef-9d3c-136c1a046ca5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '455fc9e4f0234ff7bf8834e9dcc5392c', 'request_id': 'req-25cbdce7-23d7-48ef-9d3c-136c1a046ca5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:58.488737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-25cbdce7-23d7-48ef-9d3c-136c1a046ca5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '455fc9e4f0234ff7bf8834e9dcc5392c', 'request_id': 'req-25cbdce7-23d7-48ef-9d3c-136c1a046ca5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:58.503830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-25cbdce7-23d7-48ef-9d3c-136c1a046ca5 admin admin] The token cache is being invalidate because project 455fc9e4f0234ff7bf8834e9dcc5392c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:06:58.504133 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-25cbdce7-23d7-48ef-9d3c-136c1a046ca5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:58.504820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-25cbdce7-23d7-48ef-9d3c-136c1a046ca5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:06:58.506383 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1237/2464] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:06:57 2022] DELETE /identity/v3/projects/455fc9e4f0234ff7bf8834e9dcc5392c => generated 0 bytes in 561 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:58.511993 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d638e-e338-4a6e-a1a7-b84e4688a0f0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:58.512692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d638e-e338-4a6e-a1a7-b84e4688a0f0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:58.512692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d638e-e338-4a6e-a1a7-b84e4688a0f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:58.529643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:58.546070 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d01d638e-e338-4a6e-a1a7-b84e4688a0f0 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:58.548296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d01d638e-e338-4a6e-a1a7-b84e4688a0f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:58.654801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1228/2465] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:58.690333 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b1060-a76b-4697-acfc-728c86f410c2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:58.690935 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b1060-a76b-4697-acfc-728c86f410c2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:58.691244 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b1060-a76b-4697-acfc-728c86f410c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:58.715615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-da2b1060-a76b-4697-acfc-728c86f410c2 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:58.716931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-da2b1060-a76b-4697-acfc-728c86f410c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:58.743066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1238/2466] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:06:58.747823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d638e-e338-4a6e-a1a7-b84e4688a0f0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:58.758173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d638e-e338-4a6e-a1a7-b84e4688a0f0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:58.766267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d5e36-445b-4932-af46-2df547b31e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:58.767318 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d5e36-445b-4932-af46-2df547b31e68 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:58.767846 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d5e36-445b-4932-af46-2df547b31e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:58.768241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72d5e36-445b-4932-af46-2df547b31e68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:58.769134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d5e36-445b-4932-af46-2df547b31e68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:06:58.769581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d5e36-445b-4932-af46-2df547b31e68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:58.771801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72d5e36-445b-4932-af46-2df547b31e68 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:58.799380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1229/2467] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:06:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:06:58.804431 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b1060-a76b-4697-acfc-728c86f410c2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:58.818606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b1060-a76b-4697-acfc-728c86f410c2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:58.826025 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a13aa-87c5-4db2-8351-a2176234a5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:58.826661 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a13aa-87c5-4db2-8351-a2176234a5a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:58.829897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a13aa-87c5-4db2-8351-a2176234a5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:58.829897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a13aa-87c5-4db2-8351-a2176234a5a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:58.829897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a13aa-87c5-4db2-8351-a2176234a5a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:58.831186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a13aa-87c5-4db2-8351-a2176234a5a9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:59.132095 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1239/2468] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:06:58 2022] POST /identity/v3/projects => generated 367 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:59.136684 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d5e36-445b-4932-af46-2df547b31e68 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:59.149982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72d5e36-445b-4932-af46-2df547b31e68 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:59.225940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa4245-36ed-40b4-bfdf-8b741e9e8ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:59.226467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa4245-36ed-40b4-bfdf-8b741e9e8ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:59.226745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa4245-36ed-40b4-bfdf-8b741e9e8ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:59.227017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aa4245-36ed-40b4-bfdf-8b741e9e8ea9 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:59.227841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa4245-36ed-40b4-bfdf-8b741e9e8ea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:59.229926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aa4245-36ed-40b4-bfdf-8b741e9e8ea9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:59.366491 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1230/2469] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:06:59 2022] POST /identity/v3/users => generated 284 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:06:59.368220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a13aa-87c5-4db2-8351-a2176234a5a9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:59.380511 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:06:59.392567 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a13aa-87c5-4db2-8351-a2176234a5a9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:59.398999 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77923b7-84ee-471e-9141-ff7a495d3cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:59.399617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77923b7-84ee-471e-9141-ff7a495d3cf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:59.400193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77923b7-84ee-471e-9141-ff7a495d3cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:59.400750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77923b7-84ee-471e-9141-ff7a495d3cf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:59.401407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77923b7-84ee-471e-9141-ff7a495d3cf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:59.403518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77923b7-84ee-471e-9141-ff7a495d3cf6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:59.421115 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1240/2470] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:59.425861 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa4245-36ed-40b4-bfdf-8b741e9e8ea9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:59.448290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aa4245-36ed-40b4-bfdf-8b741e9e8ea9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:59.461097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d775d1-55fb-42a4-8aaf-b003a853b9dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:59.462084 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d775d1-55fb-42a4-8aaf-b003a853b9dc admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:59.462549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d775d1-55fb-42a4-8aaf-b003a853b9dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:59.463108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d775d1-55fb-42a4-8aaf-b003a853b9dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:59.464320 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d775d1-55fb-42a4-8aaf-b003a853b9dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:59.467302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d775d1-55fb-42a4-8aaf-b003a853b9dc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:59.491859 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1231/2471] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:06:59 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:06:59.493421 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77923b7-84ee-471e-9141-ff7a495d3cf6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:06:59.509861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77923b7-84ee-471e-9141-ff7a495d3cf6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:06:59.518289 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f484d5e-ce4f-4abb-bd39-23f1a4109261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:06:59.519355 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f484d5e-ce4f-4abb-bd39-23f1a4109261 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:59.519889 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f484d5e-ce4f-4abb-bd39-23f1a4109261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:59.520323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f484d5e-ce4f-4abb-bd39-23f1a4109261 admin admin] PATH_INFO: `/v3/projects/268f5153fb244a7e9e98936909044b1d/users/c6d8eb5655c84547bc2e261bf948802a/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:59.546329 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f484d5e-ce4f-4abb-bd39-23f1a4109261 admin admin] RBAC: Authorizing `identity:create_grant(project_id=268f5153fb244a7e9e98936909044b1d, user_id=c6d8eb5655c84547bc2e261bf948802a, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:06:59.550841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f484d5e-ce4f-4abb-bd39-23f1a4109261 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:06:59.701528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1241/2472] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:06:59 2022] PUT /identity/v3/projects/268f5153fb244a7e9e98936909044b1d/users/c6d8eb5655c84547bc2e261bf948802a/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:06:59.725009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b1f08-4c2f-4365-a312-c0b6825606e7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:06:59.735308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b1f08-4c2f-4365-a312-c0b6825606e7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:06:59.735308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b1f08-4c2f-4365-a312-c0b6825606e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:06:59.769598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3d1b1f08-4c2f-4365-a312-c0b6825606e7 None None] MFA Rules not processed for user `c6d8eb5655c84547bc2e261bf948802a`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:06:59.769598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3d1b1f08-4c2f-4365-a312-c0b6825606e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:06:59.872078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1232/2473] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:06:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:00.745864 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:00.808907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:01.608958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:02.133966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:02.202988 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:02.666164 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:03.631814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:03.793242 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:04.174544 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:05.070544 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:05.198151 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:06.272312 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:06.345750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:06.911308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:07.001891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:07.517816 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:07.946068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:08.322457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:08.574265 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:08.704739 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:09.274088 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:09.332066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:09.644400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:09.722629 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:10.112170 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:10.218583 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:10.449706 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:10.757073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:11.104102 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:11.450562 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:11.660303 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:11.708885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f484d5e-ce4f-4abb-bd39-23f1a4109261 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:11.712448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6f484d5e-ce4f-4abb-bd39-23f1a4109261 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:11.816961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f484d5e-ce4f-4abb-bd39-23f1a4109261 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:11.824938 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9703317-5767-4ee9-96e8-04e410160009 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6dee005fc3d4fb593a1771e7851c20a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:11.826519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9703317-5767-4ee9-96e8-04e410160009 service glance] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:11.827319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9703317-5767-4ee9-96e8-04e410160009 service glance] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:11.828681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9703317-5767-4ee9-96e8-04e410160009 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:11.845484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9703317-5767-4ee9-96e8-04e410160009 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:11.849752 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9703317-5767-4ee9-96e8-04e410160009 service glance] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:11.868328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1242/2474] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:07:12.215285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:12.288291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:13.842643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:14.232202 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:15.071886 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:15.216015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:15.323769 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:15.494355 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:15.650399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:15.714647 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:16.110148 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:16.258378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:16.418480 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:16.460328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:16.643312 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:16.671418 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:16.761438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:16.820156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:16.882248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:17.083070 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:17.139909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:17.231257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:17.351198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:17.394801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:17.636039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:17.746112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:17.933357 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:18.060351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:18.093151 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:18.214409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:18.248669 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:18.921557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:19.096792 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:19.603858 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:19.703746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:20.102541 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:20.135931 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:20.385970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:20.438375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:20.619917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:20.718454 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:20.854319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:20.900177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:21.281269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:21.350679 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:21.854317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:21.907557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:22.078574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:22.335925 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:22.362871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:22.535448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:22.700854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:22.997214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:23.175278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:28.741578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a65955-ed56-4293-82e5-84789fbf7a16 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:28.742137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a65955-ed56-4293-82e5-84789fbf7a16 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:28.742312 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a65955-ed56-4293-82e5-84789fbf7a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:28.764735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-89a65955-ed56-4293-82e5-84789fbf7a16 None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:28.767025 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-89a65955-ed56-4293-82e5-84789fbf7a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:28.813262 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:28.839050 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1233/2475] 10.222.0.67 () {58 vars in 967 bytes} [Mon Sep 26 18:07:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:28.854465 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9703317-5767-4ee9-96e8-04e410160009 service glance] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:28.855895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d9703317-5767-4ee9-96e8-04e410160009 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:28.936507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9703317-5767-4ee9-96e8-04e410160009 service glance] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:28.943917 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd177e0f-0823-47f6-a9ed-d5a56a270a82 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:28.946793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd177e0f-0823-47f6-a9ed-d5a56a270a82 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:28.946793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd177e0f-0823-47f6-a9ed-d5a56a270a82 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:28.946793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd177e0f-0823-47f6-a9ed-d5a56a270a82 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:28.955937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd177e0f-0823-47f6-a9ed-d5a56a270a82 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:28.957597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd177e0f-0823-47f6-a9ed-d5a56a270a82 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:28.967369 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1243/2476] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:07:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:07:32.435815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a65955-ed56-4293-82e5-84789fbf7a16 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:32.439531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-89a65955-ed56-4293-82e5-84789fbf7a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:32.539690 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a65955-ed56-4293-82e5-84789fbf7a16 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:32.544924 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19ed92-b25e-4ce6-9103-011733c9bae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:32.545559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19ed92-b25e-4ce6-9103-011733c9bae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:32.545837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19ed92-b25e-4ce6-9103-011733c9bae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:32.546110 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19ed92-b25e-4ce6-9103-011733c9bae6 admin admin] PATH_INFO: `/v3/users/349bc0a30f324816804e596c937e3da0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:32.547545 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19ed92-b25e-4ce6-9103-011733c9bae6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=349bc0a30f324816804e596c937e3da0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:32.549657 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c19ed92-b25e-4ce6-9103-011733c9bae6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:32.823639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c19ed92-b25e-4ce6-9103-011733c9bae6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '349bc0a30f324816804e596c937e3da0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:32.892639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c19ed92-b25e-4ce6-9103-011733c9bae6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '349bc0a30f324816804e596c937e3da0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:32.893330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c19ed92-b25e-4ce6-9103-011733c9bae6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '349bc0a30f324816804e596c937e3da0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:32.893995 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c19ed92-b25e-4ce6-9103-011733c9bae6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '349bc0a30f324816804e596c937e3da0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:32.903072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6c19ed92-b25e-4ce6-9103-011733c9bae6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '349bc0a30f324816804e596c937e3da0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:32.922876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1234/2477] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:07:32 2022] DELETE /identity/v3/users/349bc0a30f324816804e596c937e3da0 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:33.255956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd177e0f-0823-47f6-a9ed-d5a56a270a82 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:33.272821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd177e0f-0823-47f6-a9ed-d5a56a270a82 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:33.355668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589ae74-e3ce-4696-b490-394772169dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:33.356311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589ae74-e3ce-4696-b490-394772169dd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:33.356591 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589ae74-e3ce-4696-b490-394772169dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:33.356863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589ae74-e3ce-4696-b490-394772169dd1 admin admin] PATH_INFO: `/v3/projects/31e42dd8f6ff445cb7d44b0b0ce94fc2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:33.358093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589ae74-e3ce-4696-b490-394772169dd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31e42dd8f6ff445cb7d44b0b0ce94fc2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:33.360815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589ae74-e3ce-4696-b490-394772169dd1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:33.494170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3589ae74-e3ce-4696-b490-394772169dd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31e42dd8f6ff445cb7d44b0b0ce94fc2', 'request_id': 'req-3589ae74-e3ce-4696-b490-394772169dd1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:33.501265 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3589ae74-e3ce-4696-b490-394772169dd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31e42dd8f6ff445cb7d44b0b0ce94fc2', 'request_id': 'req-3589ae74-e3ce-4696-b490-394772169dd1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:33.603204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3589ae74-e3ce-4696-b490-394772169dd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31e42dd8f6ff445cb7d44b0b0ce94fc2', 'request_id': 'req-3589ae74-e3ce-4696-b490-394772169dd1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:33.617069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3589ae74-e3ce-4696-b490-394772169dd1 admin admin] The token cache is being invalidate because project 31e42dd8f6ff445cb7d44b0b0ce94fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:07:33.617347 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3589ae74-e3ce-4696-b490-394772169dd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:33.618049 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3589ae74-e3ce-4696-b490-394772169dd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:07:33.619819 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1244/2478] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:07:33 2022] DELETE /identity/v3/projects/31e42dd8f6ff445cb7d44b0b0ce94fc2 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:07:42.918079 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=57,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:43.525760 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f6edb-08bb-4d0b-baeb-1a551a585df9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:43.525954 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f6edb-08bb-4d0b-baeb-1a551a585df9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:43.526472 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20f6edb-08bb-4d0b-baeb-1a551a585df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:43.557610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b20f6edb-08bb-4d0b-baeb-1a551a585df9 None None] MFA Rules not processed for user `0bede3a601f941e7abc9fd16983679b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:43.558872 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b20f6edb-08bb-4d0b-baeb-1a551a585df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:43.613431 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=85,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:43.618862 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1235/2479] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:07:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:48.597101 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589ae74-e3ce-4696-b490-394772169dd1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:48.598624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3589ae74-e3ce-4696-b490-394772169dd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:48.665310 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589ae74-e3ce-4696-b490-394772169dd1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:48.674981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c76a11-40de-466d-8d1a-80dfe504866a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:48.675680 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c76a11-40de-466d-8d1a-80dfe504866a service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:48.676002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c76a11-40de-466d-8d1a-80dfe504866a service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:48.676276 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c76a11-40de-466d-8d1a-80dfe504866a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:48.678090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b2c76a11-40de-466d-8d1a-80dfe504866a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:48.752977 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c76a11-40de-466d-8d1a-80dfe504866a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:48.754432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c76a11-40de-466d-8d1a-80dfe504866a service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:48.766091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1245/2480] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:07:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:07:53.611082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=25 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:07:55.100433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ef0a0-82e4-4586-98cc-59cff95efff3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:55.101340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ef0a0-82e4-4586-98cc-59cff95efff3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:55.101340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131ef0a0-82e4-4586-98cc-59cff95efff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:55.128098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-131ef0a0-82e4-4586-98cc-59cff95efff3 None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:07:55.130006 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-131ef0a0-82e4-4586-98cc-59cff95efff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:55.212171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1236/2481] 10.222.0.67 () {58 vars in 1009 bytes} [Mon Sep 26 18:07:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:07:55.229828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c76a11-40de-466d-8d1a-80dfe504866a service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:07:55.231614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b2c76a11-40de-466d-8d1a-80dfe504866a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:07:55.319341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c76a11-40de-466d-8d1a-80dfe504866a service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:07:55.325073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129daf57-415b-4205-b2e7-07cc83fe82eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:07:55.325827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129daf57-415b-4205-b2e7-07cc83fe82eb service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:07:55.326204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129daf57-415b-4205-b2e7-07cc83fe82eb service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:07:55.326576 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129daf57-415b-4205-b2e7-07cc83fe82eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:07:55.341639 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129daf57-415b-4205-b2e7-07cc83fe82eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:07:55.342761 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129daf57-415b-4205-b2e7-07cc83fe82eb service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:07:55.354992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1246/2482] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:07:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:08:00.497584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ef0a0-82e4-4586-98cc-59cff95efff3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:00.499002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-131ef0a0-82e4-4586-98cc-59cff95efff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:00.643684 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131ef0a0-82e4-4586-98cc-59cff95efff3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:00.665934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e286-e005-4238-9337-1c7010571521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:00.666876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e286-e005-4238-9337-1c7010571521 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:00.667157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e286-e005-4238-9337-1c7010571521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:00.667434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef9e286-e005-4238-9337-1c7010571521 admin admin] PATH_INFO: `/v3/users/c6d8eb5655c84547bc2e261bf948802a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:00.673468 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9e286-e005-4238-9337-1c7010571521 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6d8eb5655c84547bc2e261bf948802a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:00.680987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef9e286-e005-4238-9337-1c7010571521 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:00.896929 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cef9e286-e005-4238-9337-1c7010571521 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6d8eb5655c84547bc2e261bf948802a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:00.987083 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cef9e286-e005-4238-9337-1c7010571521 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6d8eb5655c84547bc2e261bf948802a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:00.988413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cef9e286-e005-4238-9337-1c7010571521 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6d8eb5655c84547bc2e261bf948802a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:00.989495 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cef9e286-e005-4238-9337-1c7010571521 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6d8eb5655c84547bc2e261bf948802a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:01.006315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cef9e286-e005-4238-9337-1c7010571521 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6d8eb5655c84547bc2e261bf948802a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:01.041168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1237/2483] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:08:00 2022] DELETE /identity/v3/users/c6d8eb5655c84547bc2e261bf948802a => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:08:01.046391 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b54b04-3e12-47ae-b2ff-52967b018df5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:01.047019 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b54b04-3e12-47ae-b2ff-52967b018df5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:01.047307 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b54b04-3e12-47ae-b2ff-52967b018df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:01.099169 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-68b54b04-3e12-47ae-b2ff-52967b018df5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:01.103052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-68b54b04-3e12-47ae-b2ff-52967b018df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:01.178600 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:01.260811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1247/2484] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:08:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:08:01.272714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e286-e005-4238-9337-1c7010571521 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:01.274395 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cef9e286-e005-4238-9337-1c7010571521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:01.345381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:01.393841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef9e286-e005-4238-9337-1c7010571521 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:01.400507 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131fe55-ed62-4d46-b2dd-5d759918a8ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:01.402094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131fe55-ed62-4d46-b2dd-5d759918a8ca service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:01.402369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131fe55-ed62-4d46-b2dd-5d759918a8ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:01.402648 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131fe55-ed62-4d46-b2dd-5d759918a8ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:01.416623 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6131fe55-ed62-4d46-b2dd-5d759918a8ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:01.418072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6131fe55-ed62-4d46-b2dd-5d759918a8ca service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:01.434472 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1238/2485] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:08:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:08:02.162141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b54b04-3e12-47ae-b2ff-52967b018df5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:02.164601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-68b54b04-3e12-47ae-b2ff-52967b018df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:02.201422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b54b04-3e12-47ae-b2ff-52967b018df5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:02.220379 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:02.221417 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:02.221942 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:02.222280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861 admin admin] PATH_INFO: `/v3/projects/268f5153fb244a7e9e98936909044b1d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:02.223975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=268f5153fb244a7e9e98936909044b1d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:02.226235 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:02.393492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '268f5153fb244a7e9e98936909044b1d', 'request_id': 'req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:02.404823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '268f5153fb244a7e9e98936909044b1d', 'request_id': 'req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:02.531944 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '268f5153fb244a7e9e98936909044b1d', 'request_id': 'req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:02.567952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861 admin admin] The token cache is being invalidate because project 268f5153fb244a7e9e98936909044b1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:08:02.568438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:02.573122 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:08:02.591733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1248/2486] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:08:02 2022] DELETE /identity/v3/projects/268f5153fb244a7e9e98936909044b1d => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:08:11.454806 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=15 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:12.560759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:13.544212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131fe55-ed62-4d46-b2dd-5d759918a8ca service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:13.547136 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6131fe55-ed62-4d46-b2dd-5d759918a8ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:13.563516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131fe55-ed62-4d46-b2dd-5d759918a8ca service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:13.568062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:13.570516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:13.643674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4edaa97-c8f3-4aa8-aed2-f50dd4e21785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:13.643674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edaa97-c8f3-4aa8-aed2-f50dd4e21785 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:13.643674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edaa97-c8f3-4aa8-aed2-f50dd4e21785 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:13.643674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4edaa97-c8f3-4aa8-aed2-f50dd4e21785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:13.644243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f4edaa97-c8f3-4aa8-aed2-f50dd4e21785 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:13.657954 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd0dad-5ed5-42f5-a777-cbdadd0fd861 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:13.661922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861581f6-e7d3-440a-b27d-8fe110631edc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:13.662546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861581f6-e7d3-440a-b27d-8fe110631edc service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:13.662900 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861581f6-e7d3-440a-b27d-8fe110631edc service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:13.663186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861581f6-e7d3-440a-b27d-8fe110631edc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:13.668833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-861581f6-e7d3-440a-b27d-8fe110631edc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:13.733515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edaa97-c8f3-4aa8-aed2-f50dd4e21785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:13.735617 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4edaa97-c8f3-4aa8-aed2-f50dd4e21785 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:13.747733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1239/2487] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:08:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:08:13.764636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861581f6-e7d3-440a-b27d-8fe110631edc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:13.766685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861581f6-e7d3-440a-b27d-8fe110631edc service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:13.780203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1249/2488] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:08:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:08:16.191166 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7e0d8-b7dd-4518-8d3f-966ddb3396b8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:16.191717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7e0d8-b7dd-4518-8d3f-966ddb3396b8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:16.191818 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7e0d8-b7dd-4518-8d3f-966ddb3396b8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:16.193160 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1240/2489] 10.222.0.67 () {54 vars in 921 bytes} [Mon Sep 26 18:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:08:16.200163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76b971-f156-438c-9934-fa1d613c593c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:16.200721 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76b971-f156-438c-9934-fa1d613c593c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:16.200721 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec76b971-f156-438c-9934-fa1d613c593c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:16.229167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ec76b971-f156-438c-9934-fa1d613c593c None None] MFA Rules not processed for user `b9ed29a3f6be4d64bb3751f9e95eb56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:16.230834 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ec76b971-f156-438c-9934-fa1d613c593c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:16.243378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1250/2490] 10.222.0.67 () {58 vars in 1018 bytes} [Mon Sep 26 18:08:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:08:16.261785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7e0d8-b7dd-4518-8d3f-966ddb3396b8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:16.263876 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bfb7e0d8-b7dd-4518-8d3f-966ddb3396b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:16.375566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7e0d8-b7dd-4518-8d3f-966ddb3396b8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:16.381654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb26f820-afb9-4caf-aaa2-5965af105341 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6dee005fc3d4fb593a1771e7851c20a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:16.382769 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26f820-afb9-4caf-aaa2-5965af105341 service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:16.383296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26f820-afb9-4caf-aaa2-5965af105341 service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:16.383737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb26f820-afb9-4caf-aaa2-5965af105341 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:16.398019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26f820-afb9-4caf-aaa2-5965af105341 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:16.399491 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb26f820-afb9-4caf-aaa2-5965af105341 service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:16.415617 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1241/2491] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:08:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:08:16.800622 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de83a8-e100-4f1b-b862-019774b61763 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:16.800927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de83a8-e100-4f1b-b862-019774b61763 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:16.801255 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de83a8-e100-4f1b-b862-019774b61763 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:16.802648 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1251/2492] 10.222.0.67 () {54 vars in 921 bytes} [Mon Sep 26 18:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 26 18:08:16.809190 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccf652-9176-4acc-aa1a-ec7bf9d30a29 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:16.809734 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccf652-9176-4acc-aa1a-ec7bf9d30a29 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:16.810303 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bccf652-9176-4acc-aa1a-ec7bf9d30a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:16.844654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4bccf652-9176-4acc-aa1a-ec7bf9d30a29 None None] MFA Rules not processed for user `ef7e0b772a284e7da4fae7d73f34d3de`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:16.845940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4bccf652-9176-4acc-aa1a-ec7bf9d30a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:16.854168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1242/2493] 10.222.0.67 () {58 vars in 1018 bytes} [Mon Sep 26 18:08:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:08:16.870650 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de83a8-e100-4f1b-b862-019774b61763 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:16.873946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a9de83a8-e100-4f1b-b862-019774b61763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:16.898210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de83a8-e100-4f1b-b862-019774b61763 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:16.902720 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8facaa0-7ad1-401d-8498-b7282f02e800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:16.903470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8facaa0-7ad1-401d-8498-b7282f02e800 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:16.903872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8facaa0-7ad1-401d-8498-b7282f02e800 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:16.904126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8facaa0-7ad1-401d-8498-b7282f02e800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:16.916256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8facaa0-7ad1-401d-8498-b7282f02e800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:16.917791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8facaa0-7ad1-401d-8498-b7282f02e800 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:16.932198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1252/2494] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:08:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:08:17.039014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb00ea-91a4-428f-a36e-1634666dbc4e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:17.039324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb00ea-91a4-428f-a36e-1634666dbc4e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:17.039633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb00ea-91a4-428f-a36e-1634666dbc4e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:17.042577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1243/2495] 10.222.0.67 () {54 vars in 921 bytes} [Mon Sep 26 18:08:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:08:17.049866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d2d24-21d9-4d9b-8fca-01327e3cfe54 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:17.053269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d2d24-21d9-4d9b-8fca-01327e3cfe54 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:17.053269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93d2d24-21d9-4d9b-8fca-01327e3cfe54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:17.089716 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b93d2d24-21d9-4d9b-8fca-01327e3cfe54 None None] MFA Rules not processed for user `b9ed29a3f6be4d64bb3751f9e95eb56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:17.090978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b93d2d24-21d9-4d9b-8fca-01327e3cfe54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:17.102477 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1253/2496] 10.222.0.67 () {58 vars in 1018 bytes} [Mon Sep 26 18:08:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:08:17.116499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb00ea-91a4-428f-a36e-1634666dbc4e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:17.130409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb00ea-91a4-428f-a36e-1634666dbc4e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:17.134341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59f05ef-3f1d-4d19-95cc-5de066ebba6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:17.135029 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f05ef-3f1d-4d19-95cc-5de066ebba6a service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:17.135503 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f05ef-3f1d-4d19-95cc-5de066ebba6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:17.135835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59f05ef-3f1d-4d19-95cc-5de066ebba6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:17.150529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f05ef-3f1d-4d19-95cc-5de066ebba6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:17.152039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59f05ef-3f1d-4d19-95cc-5de066ebba6a service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:17.164019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1244/2497] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:08:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:08:17.670092 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499ab5b-5ea5-4c26-906b-00cb1e77fc10 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:17.676528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499ab5b-5ea5-4c26-906b-00cb1e77fc10 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:17.676528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499ab5b-5ea5-4c26-906b-00cb1e77fc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:17.700221 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3499ab5b-5ea5-4c26-906b-00cb1e77fc10 None None] MFA Rules not processed for user `97ed136f68c540abbdaf04cbe34a682e`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:17.701566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3499ab5b-5ea5-4c26-906b-00cb1e77fc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:17.761871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1254/2498] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:08:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:08:25.022765 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4f9be-771b-4666-b9ae-5d6a3eaa8d5c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:25.023320 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4f9be-771b-4666-b9ae-5d6a3eaa8d5c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:25.023320 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4f9be-771b-4666-b9ae-5d6a3eaa8d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:25.044849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e8b4f9be-771b-4666-b9ae-5d6a3eaa8d5c None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:25.046676 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e8b4f9be-771b-4666-b9ae-5d6a3eaa8d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:25.068888 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:25.112025 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1245/2499] 10.222.0.67 () {58 vars in 1009 bytes} [Mon Sep 26 18:08:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:08:25.124946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499ab5b-5ea5-4c26-906b-00cb1e77fc10 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:25.126377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3499ab5b-5ea5-4c26-906b-00cb1e77fc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:25.150414 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:25.210778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3499ab5b-5ea5-4c26-906b-00cb1e77fc10 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:25.219148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e0b5d-700c-4715-9bae-b57d5fa3c365 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:25.219911 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e0b5d-700c-4715-9bae-b57d5fa3c365 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:25.220629 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e0b5d-700c-4715-9bae-b57d5fa3c365 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:25.221119 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913e0b5d-700c-4715-9bae-b57d5fa3c365 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:25.233433 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e0b5d-700c-4715-9bae-b57d5fa3c365 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:25.235700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913e0b5d-700c-4715-9bae-b57d5fa3c365 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:25.245200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1255/2500] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:08:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 18:08:35.127637 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=14 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:35.241160 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=23 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:08:50.978643 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5cb5a-8342-40e7-87bf-381993b23e93 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:50.979596 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5cb5a-8342-40e7-87bf-381993b23e93 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:50.979792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc5cb5a-8342-40e7-87bf-381993b23e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:51.018772 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-dcc5cb5a-8342-40e7-87bf-381993b23e93 None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:08:51.020549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dcc5cb5a-8342-40e7-87bf-381993b23e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:51.031673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1246/2501] 10.222.0.67 () {58 vars in 967 bytes} [Mon Sep 26 18:08:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:08:51.045092 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e0b5d-700c-4715-9bae-b57d5fa3c365 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:51.055992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913e0b5d-700c-4715-9bae-b57d5fa3c365 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:51.061371 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bd9d5-dadf-4729-95f3-951f56d14c63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:51.062110 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bd9d5-dadf-4729-95f3-951f56d14c63 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:51.062456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bd9d5-dadf-4729-95f3-951f56d14c63 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:51.062809 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bd9d5-dadf-4729-95f3-951f56d14c63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:51.073829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7bd9d5-dadf-4729-95f3-951f56d14c63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:51.076073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7bd9d5-dadf-4729-95f3-951f56d14c63 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:51.085863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1256/2502] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:08:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:08:54.732332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5cb5a-8342-40e7-87bf-381993b23e93 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:08:54.734381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dcc5cb5a-8342-40e7-87bf-381993b23e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:54.750772 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc5cb5a-8342-40e7-87bf-381993b23e93 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:08:54.756766 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70512de8-3bf2-4a08-8107-5c2db3beff99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:08:54.757367 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70512de8-3bf2-4a08-8107-5c2db3beff99 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:08:54.757647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70512de8-3bf2-4a08-8107-5c2db3beff99 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:08:54.757920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70512de8-3bf2-4a08-8107-5c2db3beff99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:08:54.759588 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-70512de8-3bf2-4a08-8107-5c2db3beff99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:08:54.829407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70512de8-3bf2-4a08-8107-5c2db3beff99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:08:54.831892 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70512de8-3bf2-4a08-8107-5c2db3beff99 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:08:54.841663 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1247/2503] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:08:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:01.082205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:04.836834 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=31 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:05.269201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bd9d5-dadf-4729-95f3-951f56d14c63 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:05.270605 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-df7bd9d5-dadf-4729-95f3-951f56d14c63 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:05.331252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bd9d5-dadf-4729-95f3-951f56d14c63 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:05.335334 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69167b-4a7b-4ea4-a25a-4d7f254907e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:05.335994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69167b-4a7b-4ea4-a25a-4d7f254907e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:05.336274 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69167b-4a7b-4ea4-a25a-4d7f254907e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:05.336546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69167b-4a7b-4ea4-a25a-4d7f254907e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:05.338271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2d69167b-4a7b-4ea4-a25a-4d7f254907e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:05.351418 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69167b-4a7b-4ea4-a25a-4d7f254907e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:05.354104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69167b-4a7b-4ea4-a25a-4d7f254907e7 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:05.363586 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1257/2504] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:05 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:13.994765 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70512de8-3bf2-4a08-8107-5c2db3beff99 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:14.008633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70512de8-3bf2-4a08-8107-5c2db3beff99 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:14.015361 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f1d86-775f-4732-bf76-aaf7a59990d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:14.016036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f1d86-775f-4732-bf76-aaf7a59990d2 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:14.016309 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f1d86-775f-4732-bf76-aaf7a59990d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:14.016615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f1d86-775f-4732-bf76-aaf7a59990d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:14.019130 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-052f1d86-775f-4732-bf76-aaf7a59990d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:14.033225 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f1d86-775f-4732-bf76-aaf7a59990d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:14.035217 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f1d86-775f-4732-bf76-aaf7a59990d2 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:14.047190 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1248/2505] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:15.358308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=26 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:18.476791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69167b-4a7b-4ea4-a25a-4d7f254907e7 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:18.488118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69167b-4a7b-4ea4-a25a-4d7f254907e7 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:18.492314 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a3a5e-9c74-4a0c-8781-8b0c63248c77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:18.492938 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a3a5e-9c74-4a0c-8781-8b0c63248c77 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:18.493397 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a3a5e-9c74-4a0c-8781-8b0c63248c77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:18.493627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a3a5e-9c74-4a0c-8781-8b0c63248c77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:18.495442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-710a3a5e-9c74-4a0c-8781-8b0c63248c77 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:18.506130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a3a5e-9c74-4a0c-8781-8b0c63248c77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:18.507698 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a3a5e-9c74-4a0c-8781-8b0c63248c77 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:18.517342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1258/2506] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:23.885507 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb5e9-221a-4eb8-835b-374869784792 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:23.886071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb5e9-221a-4eb8-835b-374869784792 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:23.886071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb5e9-221a-4eb8-835b-374869784792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:23.913181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8fdcb5e9-221a-4eb8-835b-374869784792 None None] MFA Rules not processed for user `0bede3a601f941e7abc9fd16983679b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:23.914649 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8fdcb5e9-221a-4eb8-835b-374869784792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:23.926779 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1249/2507] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:09:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:09:24.260861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a3a5e-9c74-4a0c-8781-8b0c63248c77 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:24.272162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a3a5e-9c74-4a0c-8781-8b0c63248c77 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:24.275867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab0fb0-4964-402d-ae66-678a607e855c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:24.276477 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab0fb0-4964-402d-ae66-678a607e855c service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:24.276758 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab0fb0-4964-402d-ae66-678a607e855c service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:24.277030 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab0fb0-4964-402d-ae66-678a607e855c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:24.278738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d7ab0fb0-4964-402d-ae66-678a607e855c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:24.289576 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab0fb0-4964-402d-ae66-678a607e855c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:24.290992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab0fb0-4964-402d-ae66-678a607e855c service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:24.300414 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1259/2508] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:24.845862 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07d37a-c242-4ae1-8aad-da6a85440464 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:24.846189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07d37a-c242-4ae1-8aad-da6a85440464 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:24.846503 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b07d37a-c242-4ae1-8aad-da6a85440464 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:24.848002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1250/2509] 10.222.0.67 () {54 vars in 961 bytes} [Mon Sep 26 18:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:09:24.855482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd9cd9-777b-4a91-968a-c72565fc87a6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:24.855945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd9cd9-777b-4a91-968a-c72565fc87a6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:24.856323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd9cd9-777b-4a91-968a-c72565fc87a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:24.879768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-61fd9cd9-777b-4a91-968a-c72565fc87a6 None None] MFA Rules not processed for user `22a512e0a64e420abf15b2150ccc2b32`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:24.881113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-61fd9cd9-777b-4a91-968a-c72565fc87a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:24.955889 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1260/2510] 10.222.0.67 () {58 vars in 1058 bytes} [Mon Sep 26 18:09:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:09:24.964661 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07d37a-c242-4ae1-8aad-da6a85440464 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:24.988414 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b07d37a-c242-4ae1-8aad-da6a85440464 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:24.992374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c282f-8378-4531-98a6-98a88deb6bf2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:24.992991 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c282f-8378-4531-98a6-98a88deb6bf2 service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:24.993275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c282f-8378-4531-98a6-98a88deb6bf2 service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:24.993723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351c282f-8378-4531-98a6-98a88deb6bf2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:24.996116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-351c282f-8378-4531-98a6-98a88deb6bf2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:25.010056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c282f-8378-4531-98a6-98a88deb6bf2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:25.011766 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351c282f-8378-4531-98a6-98a88deb6bf2 service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:25.023328 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1251/2511] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:09:33.895570 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd9cd9-777b-4a91-968a-c72565fc87a6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:33.905870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd9cd9-777b-4a91-968a-c72565fc87a6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:33.909852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ace9c4-ad4e-46b3-a105-df2bbfc1f503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:33.910462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ace9c4-ad4e-46b3-a105-df2bbfc1f503 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:33.910744 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ace9c4-ad4e-46b3-a105-df2bbfc1f503 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:33.911010 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ace9c4-ad4e-46b3-a105-df2bbfc1f503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:33.912764 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-78ace9c4-ad4e-46b3-a105-df2bbfc1f503 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:33.985597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ace9c4-ad4e-46b3-a105-df2bbfc1f503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:33.987711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ace9c4-ad4e-46b3-a105-df2bbfc1f503 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:33.998854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1261/2512] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:35.018368 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=15 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:38.303223 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c282f-8378-4531-98a6-98a88deb6bf2 service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:38.315573 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351c282f-8378-4531-98a6-98a88deb6bf2 service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:38.322860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe419f-3149-4640-8c28-a002a8ae5e8d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:38.323492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe419f-3149-4640-8c28-a002a8ae5e8d service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:38.323803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe419f-3149-4640-8c28-a002a8ae5e8d service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:38.324074 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbe419f-3149-4640-8c28-a002a8ae5e8d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:38.325897 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5fbe419f-3149-4640-8c28-a002a8ae5e8d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:38.340440 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe419f-3149-4640-8c28-a002a8ae5e8d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:38.341186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbe419f-3149-4640-8c28-a002a8ae5e8d service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:38.353627 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1252/2513] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:38.442304 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ace9c4-ad4e-46b3-a105-df2bbfc1f503 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:38.442304 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-78ace9c4-ad4e-46b3-a105-df2bbfc1f503 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:38.457036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ace9c4-ad4e-46b3-a105-df2bbfc1f503 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:38.461946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f193713-959d-4341-b5e7-2a1a2b86c9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:38.462592 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f193713-959d-4341-b5e7-2a1a2b86c9e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:38.462875 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f193713-959d-4341-b5e7-2a1a2b86c9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:38.463150 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f193713-959d-4341-b5e7-2a1a2b86c9e1 admin admin] PATH_INFO: `/v3/users/97ed136f68c540abbdaf04cbe34a682e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:38.464285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f193713-959d-4341-b5e7-2a1a2b86c9e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97ed136f68c540abbdaf04cbe34a682e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:38.466238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f193713-959d-4341-b5e7-2a1a2b86c9e1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:38.767059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7f193713-959d-4341-b5e7-2a1a2b86c9e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97ed136f68c540abbdaf04cbe34a682e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:38.866197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7f193713-959d-4341-b5e7-2a1a2b86c9e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97ed136f68c540abbdaf04cbe34a682e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:38.866884 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7f193713-959d-4341-b5e7-2a1a2b86c9e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97ed136f68c540abbdaf04cbe34a682e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:38.867553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7f193713-959d-4341-b5e7-2a1a2b86c9e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97ed136f68c540abbdaf04cbe34a682e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:38.873181 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:38.888967 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7f193713-959d-4341-b5e7-2a1a2b86c9e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97ed136f68c540abbdaf04cbe34a682e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:38.902828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1262/2514] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:09:38 2022] DELETE /identity/v3/users/97ed136f68c540abbdaf04cbe34a682e => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:09:39.208859 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe419f-3149-4640-8c28-a002a8ae5e8d service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:39.222176 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbe419f-3149-4640-8c28-a002a8ae5e8d service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:39.284649 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8369266-dfab-4040-9fbd-da8375aba2fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:39.285278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8369266-dfab-4040-9fbd-da8375aba2fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:39.285556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8369266-dfab-4040-9fbd-da8375aba2fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:39.285824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8369266-dfab-4040-9fbd-da8375aba2fb admin admin] PATH_INFO: `/v3/projects/3a15ebd862054a64ac1d739db5adc3a2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:39.286963 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8369266-dfab-4040-9fbd-da8375aba2fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a15ebd862054a64ac1d739db5adc3a2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:39.289031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8369266-dfab-4040-9fbd-da8375aba2fb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:39.446614 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d8369266-dfab-4040-9fbd-da8375aba2fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a15ebd862054a64ac1d739db5adc3a2', 'request_id': 'req-d8369266-dfab-4040-9fbd-da8375aba2fb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:39.452844 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d8369266-dfab-4040-9fbd-da8375aba2fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a15ebd862054a64ac1d739db5adc3a2', 'request_id': 'req-d8369266-dfab-4040-9fbd-da8375aba2fb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:39.550840 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d8369266-dfab-4040-9fbd-da8375aba2fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a15ebd862054a64ac1d739db5adc3a2', 'request_id': 'req-d8369266-dfab-4040-9fbd-da8375aba2fb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:39.568657 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d8369266-dfab-4040-9fbd-da8375aba2fb admin admin] The token cache is being invalidate because project 3a15ebd862054a64ac1d739db5adc3a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:09:39.568961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d8369266-dfab-4040-9fbd-da8375aba2fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:39.569654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d8369266-dfab-4040-9fbd-da8375aba2fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:09:39.571192 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1253/2515] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:09:39 2022] DELETE /identity/v3/projects/3a15ebd862054a64ac1d739db5adc3a2 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:09:39.577471 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca52c0-b592-44ae-8796-26dfed62be5b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:39.578106 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca52c0-b592-44ae-8796-26dfed62be5b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:39.578236 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dca52c0-b592-44ae-8796-26dfed62be5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:39.601134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5dca52c0-b592-44ae-8796-26dfed62be5b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:39.602337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5dca52c0-b592-44ae-8796-26dfed62be5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:39.794632 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1263/2516] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:09:39.803259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a692118-0af1-40f2-9d5c-b5ff534540b2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:39.803549 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a692118-0af1-40f2-9d5c-b5ff534540b2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:39.803861 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a692118-0af1-40f2-9d5c-b5ff534540b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:39.827451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9a692118-0af1-40f2-9d5c-b5ff534540b2 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:39.828809 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9a692118-0af1-40f2-9d5c-b5ff534540b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:39.837633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1254/2517] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:09:39.842302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca52c0-b592-44ae-8796-26dfed62be5b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:39.854684 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dca52c0-b592-44ae-8796-26dfed62be5b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:39.859490 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd25bc-3d38-4ff5-8c7a-b360e707e83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:39.864637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd25bc-3d38-4ff5-8c7a-b360e707e83e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:39.864637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd25bc-3d38-4ff5-8c7a-b360e707e83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:39.864637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd25bc-3d38-4ff5-8c7a-b360e707e83e admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:39.864637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd25bc-3d38-4ff5-8c7a-b360e707e83e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:09:39.864637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd25bc-3d38-4ff5-8c7a-b360e707e83e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:39.864637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd25bc-3d38-4ff5-8c7a-b360e707e83e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:39.877456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1264/2518] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:09:39.882127 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a692118-0af1-40f2-9d5c-b5ff534540b2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:39.894928 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a692118-0af1-40f2-9d5c-b5ff534540b2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:39.900444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1913ddf-4629-4d06-9e9c-84b270ce678f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:39.901131 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1913ddf-4629-4d06-9e9c-84b270ce678f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:39.901481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1913ddf-4629-4d06-9e9c-84b270ce678f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:39.901821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1913ddf-4629-4d06-9e9c-84b270ce678f admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:39.902556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1913ddf-4629-4d06-9e9c-84b270ce678f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:39.904672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1913ddf-4629-4d06-9e9c-84b270ce678f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:40.183112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1255/2519] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:09:39 2022] POST /identity/v3/projects => generated 375 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:09:40.188206 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd25bc-3d38-4ff5-8c7a-b360e707e83e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:40.198811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd25bc-3d38-4ff5-8c7a-b360e707e83e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:40.279465 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c53d0-c90b-49d1-ac26-c583ce67e12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:40.280385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c53d0-c90b-49d1-ac26-c583ce67e12a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:40.280790 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c53d0-c90b-49d1-ac26-c583ce67e12a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:40.281163 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c53d0-c90b-49d1-ac26-c583ce67e12a admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:40.282273 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c53d0-c90b-49d1-ac26-c583ce67e12a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:40.285537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c53d0-c90b-49d1-ac26-c583ce67e12a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:40.485758 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1265/2520] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:09:40 2022] POST /identity/v3/users => generated 288 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:09:40.491268 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1913ddf-4629-4d06-9e9c-84b270ce678f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:40.504466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1913ddf-4629-4d06-9e9c-84b270ce678f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:40.510864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03ca50-f506-40a6-9fb3-22afc4a7a8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:40.511974 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03ca50-f506-40a6-9fb3-22afc4a7a8e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:40.512425 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03ca50-f506-40a6-9fb3-22afc4a7a8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:40.512856 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03ca50-f506-40a6-9fb3-22afc4a7a8e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:40.513951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03ca50-f506-40a6-9fb3-22afc4a7a8e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:40.522644 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03ca50-f506-40a6-9fb3-22afc4a7a8e2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:40.528572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1256/2521] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:09:40.532599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c53d0-c90b-49d1-ac26-c583ce67e12a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:40.545571 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c53d0-c90b-49d1-ac26-c583ce67e12a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:40.552313 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8c135-ecf4-458b-934c-f056150aa799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:40.552987 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8c135-ecf4-458b-934c-f056150aa799 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:40.553269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8c135-ecf4-458b-934c-f056150aa799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:40.553545 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8c135-ecf4-458b-934c-f056150aa799 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:40.554203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa8c135-ecf4-458b-934c-f056150aa799 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:40.555891 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa8c135-ecf4-458b-934c-f056150aa799 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:40.570311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1266/2522] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:09:40.576043 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03ca50-f506-40a6-9fb3-22afc4a7a8e2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:40.588570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03ca50-f506-40a6-9fb3-22afc4a7a8e2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:40.593082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a2e15-79b9-43ad-a6fc-d06841d4ab32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:40.593754 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a2e15-79b9-43ad-a6fc-d06841d4ab32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:40.594462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a2e15-79b9-43ad-a6fc-d06841d4ab32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:40.595019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a2e15-79b9-43ad-a6fc-d06841d4ab32 admin admin] PATH_INFO: `/v3/projects/7750acf9ab4843d1a3848472e99881a1/users/4b1cdc37455c489fb45e1ce3e2421cc0/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:40.612635 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8c135-ecf4-458b-934c-f056150aa799 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:40.615922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-afa8c135-ecf4-458b-934c-f056150aa799 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:40.619843 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a2e15-79b9-43ad-a6fc-d06841d4ab32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7750acf9ab4843d1a3848472e99881a1, user_id=4b1cdc37455c489fb45e1ce3e2421cc0, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:40.631914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a2e15-79b9-43ad-a6fc-d06841d4ab32 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:40.681019 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:40.738484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8c135-ecf4-458b-934c-f056150aa799 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:40.747260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38790ede-51dd-4fea-a582-29d841df41d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:40.748119 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38790ede-51dd-4fea-a582-29d841df41d2 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:40.748500 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38790ede-51dd-4fea-a582-29d841df41d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:40.749074 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38790ede-51dd-4fea-a582-29d841df41d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:40.751245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-38790ede-51dd-4fea-a582-29d841df41d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:40.781402 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1257/2523] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:09:40 2022] PUT /identity/v3/projects/7750acf9ab4843d1a3848472e99881a1/users/4b1cdc37455c489fb45e1ce3e2421cc0/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:09:40.784466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748347b-e1e4-479d-a971-95c0b6eb4bf1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:40.784766 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748347b-e1e4-479d-a971-95c0b6eb4bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:40.785219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d748347b-e1e4-479d-a971-95c0b6eb4bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:40.786695 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38790ede-51dd-4fea-a582-29d841df41d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:40.792181 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38790ede-51dd-4fea-a582-29d841df41d2 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:40.824093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1267/2524] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:09:40.844835 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d748347b-e1e4-479d-a971-95c0b6eb4bf1 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:40.847142 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d748347b-e1e4-479d-a971-95c0b6eb4bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:40.857300 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:40.942691 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1258/2525] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:09:40.969312 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38790ede-51dd-4fea-a582-29d841df41d2 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:40.972311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-38790ede-51dd-4fea-a582-29d841df41d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:41.059103 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38790ede-51dd-4fea-a582-29d841df41d2 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:41.114712 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7155ab-fac6-4a20-9eed-3d1a552d53ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:41.115324 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7155ab-fac6-4a20-9eed-3d1a552d53ba service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:41.115740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7155ab-fac6-4a20-9eed-3d1a552d53ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:41.115958 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7155ab-fac6-4a20-9eed-3d1a552d53ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:41.125302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7155ab-fac6-4a20-9eed-3d1a552d53ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:41.130332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7155ab-fac6-4a20-9eed-3d1a552d53ba service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:41.145934 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1268/2526] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:44.959056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748347b-e1e4-479d-a971-95c0b6eb4bf1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:45.094997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d748347b-e1e4-479d-a971-95c0b6eb4bf1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:45.103900 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c19739-9b20-46d0-8c43-2c6c70d62ebb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:45.104605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c19739-9b20-46d0-8c43-2c6c70d62ebb service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:45.104926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c19739-9b20-46d0-8c43-2c6c70d62ebb service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:45.105218 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c19739-9b20-46d0-8c43-2c6c70d62ebb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:45.107253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-00c19739-9b20-46d0-8c43-2c6c70d62ebb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:45.122852 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c19739-9b20-46d0-8c43-2c6c70d62ebb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:45.124633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c19739-9b20-46d0-8c43-2c6c70d62ebb service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:45.157554 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1259/2527] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:48.307606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512424d-4b81-4cfa-b2d7-c3194f9b01c1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:48.308625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512424d-4b81-4cfa-b2d7-c3194f9b01c1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:48.308964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e512424d-4b81-4cfa-b2d7-c3194f9b01c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:48.341745 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e512424d-4b81-4cfa-b2d7-c3194f9b01c1 None None] MFA Rules not processed for user `4b1cdc37455c489fb45e1ce3e2421cc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:48.342907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e512424d-4b81-4cfa-b2d7-c3194f9b01c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:48.434683 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1269/2528] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:09:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:09:48.456158 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c19739-9b20-46d0-8c43-2c6c70d62ebb service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:09:48.474928 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c19739-9b20-46d0-8c43-2c6c70d62ebb service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:09:48.483764 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e95b71-9357-4e91-aee1-933bbb5e2f8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:09:48.484081 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e95b71-9357-4e91-aee1-933bbb5e2f8c service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:48.484485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e95b71-9357-4e91-aee1-933bbb5e2f8c service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:48.484831 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e95b71-9357-4e91-aee1-933bbb5e2f8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:48.495629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e95b71-9357-4e91-aee1-933bbb5e2f8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:09:48.497412 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e95b71-9357-4e91-aee1-933bbb5e2f8c service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:09:48.508875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1260/2529] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:09:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:09:49.928275 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5bbe81-89f3-4242-948b-acf88fcd5fe1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:49.929294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5bbe81-89f3-4242-948b-acf88fcd5fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:49.929806 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5bbe81-89f3-4242-948b-acf88fcd5fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:49.963375 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2f5bbe81-89f3-4242-948b-acf88fcd5fe1 None None] MFA Rules not processed for user `612875c8fc7240e5b18fe38a034a3ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:49.964998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2f5bbe81-89f3-4242-948b-acf88fcd5fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:49.988833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:09:50.074732 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1270/2530] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:09:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:09:52.188117 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b5085-08e4-4eb7-abf3-221e9af700f8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:09:52.188890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b5085-08e4-4eb7-abf3-221e9af700f8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:09:52.189458 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b5085-08e4-4eb7-abf3-221e9af700f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:09:52.218468 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4e4b5085-08e4-4eb7-abf3-221e9af700f8 None None] MFA Rules not processed for user `4b1cdc37455c489fb45e1ce3e2421cc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:09:52.220141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4e4b5085-08e4-4eb7-abf3-221e9af700f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:09:52.229476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1261/2531] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:09:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:10:00.277690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=18 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:10:02.210803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=64 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:10:14.739914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5bbe81-89f3-4242-948b-acf88fcd5fe1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:10:14.761758 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5bbe81-89f3-4242-948b-acf88fcd5fe1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:10:14.763514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9db283-b85a-4cef-a8c8-87b862563e33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:10:14.764470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9db283-b85a-4cef-a8c8-87b862563e33 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:10:14.764924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9db283-b85a-4cef-a8c8-87b862563e33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:10:14.765332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9db283-b85a-4cef-a8c8-87b862563e33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:10:14.768869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2d9db283-b85a-4cef-a8c8-87b862563e33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:10:14.780902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9db283-b85a-4cef-a8c8-87b862563e33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:10:14.782349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9db283-b85a-4cef-a8c8-87b862563e33 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:10:14.792725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1271/2532] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:10:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:10:24.788115 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:10:52.762413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b5085-08e4-4eb7-abf3-221e9af700f8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:10:52.764240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4e4b5085-08e4-4eb7-abf3-221e9af700f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:10:52.782091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b5085-08e4-4eb7-abf3-221e9af700f8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:10:52.788244 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad65b74-44d1-493c-99f3-c769ff4d97e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:10:52.788856 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad65b74-44d1-493c-99f3-c769ff4d97e2 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:10:52.789141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad65b74-44d1-493c-99f3-c769ff4d97e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:10:52.789442 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad65b74-44d1-493c-99f3-c769ff4d97e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:10:52.791320 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fad65b74-44d1-493c-99f3-c769ff4d97e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:10:52.803019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad65b74-44d1-493c-99f3-c769ff4d97e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:10:52.804640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad65b74-44d1-493c-99f3-c769ff4d97e2 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:10:52.815021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1262/2533] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:10:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:10:54.686692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9db283-b85a-4cef-a8c8-87b862563e33 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:10:54.695592 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9db283-b85a-4cef-a8c8-87b862563e33 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:10:54.699848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69f43a-3f58-4ee8-b684-a179008fb95a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:10:54.700450 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69f43a-3f58-4ee8-b684-a179008fb95a service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:10:54.700738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69f43a-3f58-4ee8-b684-a179008fb95a service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:10:54.701012 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc69f43a-3f58-4ee8-b684-a179008fb95a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:10:54.702762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dc69f43a-3f58-4ee8-b684-a179008fb95a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:10:54.716167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69f43a-3f58-4ee8-b684-a179008fb95a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:10:54.717566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc69f43a-3f58-4ee8-b684-a179008fb95a service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:10:54.729000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1272/2534] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:10:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:10:58.989822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad65b74-44d1-493c-99f3-c769ff4d97e2 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:10:58.991330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fad65b74-44d1-493c-99f3-c769ff4d97e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:10:59.095767 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad65b74-44d1-493c-99f3-c769ff4d97e2 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:10:59.115151 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44b80a-85e1-4a2f-8674-c401fcc6c0e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:10:59.115891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44b80a-85e1-4a2f-8674-c401fcc6c0e5 service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:10:59.116237 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44b80a-85e1-4a2f-8674-c401fcc6c0e5 service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:10:59.116581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e44b80a-85e1-4a2f-8674-c401fcc6c0e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:10:59.118462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4e44b80a-85e1-4a2f-8674-c401fcc6c0e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:10:59.131728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69f43a-3f58-4ee8-b684-a179008fb95a service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:10:59.133336 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dc69f43a-3f58-4ee8-b684-a179008fb95a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:10:59.133504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44b80a-85e1-4a2f-8674-c401fcc6c0e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:10:59.135223 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e44b80a-85e1-4a2f-8674-c401fcc6c0e5 service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:10:59.146914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1263/2535] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:10:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:10:59.147460 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc69f43a-3f58-4ee8-b684-a179008fb95a service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:10:59.152263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1337e-2b96-473a-b43f-705134dd4a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:10:59.152897 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1337e-2b96-473a-b43f-705134dd4a5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:10:59.153178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1337e-2b96-473a-b43f-705134dd4a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:10:59.153452 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1337e-2b96-473a-b43f-705134dd4a5b admin admin] PATH_INFO: `/v3/users/0bede3a601f941e7abc9fd16983679b2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:10:59.154577 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1337e-2b96-473a-b43f-705134dd4a5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bede3a601f941e7abc9fd16983679b2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:10:59.156515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f1337e-2b96-473a-b43f-705134dd4a5b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:10:59.349257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d0f1337e-2b96-473a-b43f-705134dd4a5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bede3a601f941e7abc9fd16983679b2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:59.402868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d0f1337e-2b96-473a-b43f-705134dd4a5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bede3a601f941e7abc9fd16983679b2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:59.404421 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d0f1337e-2b96-473a-b43f-705134dd4a5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bede3a601f941e7abc9fd16983679b2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:59.404421 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d0f1337e-2b96-473a-b43f-705134dd4a5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bede3a601f941e7abc9fd16983679b2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:59.426294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d0f1337e-2b96-473a-b43f-705134dd4a5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bede3a601f941e7abc9fd16983679b2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:59.439135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1273/2536] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:10:59 2022] DELETE /identity/v3/users/0bede3a601f941e7abc9fd16983679b2 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:10:59.754493 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44b80a-85e1-4a2f-8674-c401fcc6c0e5 service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:10:59.769159 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e44b80a-85e1-4a2f-8674-c401fcc6c0e5 service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:10:59.834984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d400f4f-08d7-4b44-a806-6490152a5132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:10:59.835589 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d400f4f-08d7-4b44-a806-6490152a5132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:10:59.835906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d400f4f-08d7-4b44-a806-6490152a5132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:10:59.836179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d400f4f-08d7-4b44-a806-6490152a5132 admin admin] PATH_INFO: `/v3/projects/ffa58ff5197e48709d5c90ec334b9403` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:10:59.837320 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d400f4f-08d7-4b44-a806-6490152a5132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffa58ff5197e48709d5c90ec334b9403)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:10:59.839249 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d400f4f-08d7-4b44-a806-6490152a5132 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:10:59.963171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d400f4f-08d7-4b44-a806-6490152a5132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffa58ff5197e48709d5c90ec334b9403', 'request_id': 'req-0d400f4f-08d7-4b44-a806-6490152a5132'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:10:59.970854 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d400f4f-08d7-4b44-a806-6490152a5132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffa58ff5197e48709d5c90ec334b9403', 'request_id': 'req-0d400f4f-08d7-4b44-a806-6490152a5132'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:00.054342 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d400f4f-08d7-4b44-a806-6490152a5132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffa58ff5197e48709d5c90ec334b9403', 'request_id': 'req-0d400f4f-08d7-4b44-a806-6490152a5132'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:00.069228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d400f4f-08d7-4b44-a806-6490152a5132 admin admin] The token cache is being invalidate because project ffa58ff5197e48709d5c90ec334b9403 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:11:00.069228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d400f4f-08d7-4b44-a806-6490152a5132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:00.072398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d400f4f-08d7-4b44-a806-6490152a5132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:00.072398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1264/2537] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:10:59 2022] DELETE /identity/v3/projects/ffa58ff5197e48709d5c90ec334b9403 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:11:00.076601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400480dd-2118-4df2-9a6a-e8923d0333fa None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:00.076889 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400480dd-2118-4df2-9a6a-e8923d0333fa None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:00.077165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400480dd-2118-4df2-9a6a-e8923d0333fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:00.097642 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-400480dd-2118-4df2-9a6a-e8923d0333fa None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:00.098828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-400480dd-2118-4df2-9a6a-e8923d0333fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:00.191745 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1274/2538] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:11:00.195958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef34f53-9c41-46b2-af56-9e1001b0f6c6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:00.196230 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef34f53-9c41-46b2-af56-9e1001b0f6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:00.197336 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef34f53-9c41-46b2-af56-9e1001b0f6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:00.218266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:00.226986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3ef34f53-9c41-46b2-af56-9e1001b0f6c6 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:00.230031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3ef34f53-9c41-46b2-af56-9e1001b0f6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:00.243213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1265/2539] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:00.246954 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400480dd-2118-4df2-9a6a-e8923d0333fa None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:00.259117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400480dd-2118-4df2-9a6a-e8923d0333fa None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:00.266042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882adbc-9704-4295-9aa1-e1024d43051c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:00.266759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882adbc-9704-4295-9aa1-e1024d43051c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:00.275555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882adbc-9704-4295-9aa1-e1024d43051c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:00.275555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9882adbc-9704-4295-9aa1-e1024d43051c admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:00.275555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882adbc-9704-4295-9aa1-e1024d43051c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:11:00.275555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882adbc-9704-4295-9aa1-e1024d43051c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:00.275555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9882adbc-9704-4295-9aa1-e1024d43051c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:00.283302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1275/2540] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:11:00.287891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef34f53-9c41-46b2-af56-9e1001b0f6c6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:00.299765 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef34f53-9c41-46b2-af56-9e1001b0f6c6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:00.315851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5499f-408b-44ab-ad0d-6ec755a45506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:00.315851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5499f-408b-44ab-ad0d-6ec755a45506 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:00.315851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5499f-408b-44ab-ad0d-6ec755a45506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:00.315851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a5499f-408b-44ab-ad0d-6ec755a45506 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:00.317386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5499f-408b-44ab-ad0d-6ec755a45506 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:00.317757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a5499f-408b-44ab-ad0d-6ec755a45506 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:00.502072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1266/2541] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:11:00 2022] POST /identity/v3/projects => generated 363 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:00.506345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882adbc-9704-4295-9aa1-e1024d43051c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:00.517891 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9882adbc-9704-4295-9aa1-e1024d43051c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:00.567688 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:00.580641 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df40ee-9e31-4cd6-b341-8839b2a9354a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:00.590308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df40ee-9e31-4cd6-b341-8839b2a9354a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:00.590308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df40ee-9e31-4cd6-b341-8839b2a9354a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:00.590308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df40ee-9e31-4cd6-b341-8839b2a9354a admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:00.590308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df40ee-9e31-4cd6-b341-8839b2a9354a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:00.590308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df40ee-9e31-4cd6-b341-8839b2a9354a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:00.736142 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1276/2542] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:11:00 2022] POST /identity/v3/users => generated 282 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:11:00.740645 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5499f-408b-44ab-ad0d-6ec755a45506 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:00.751633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a5499f-408b-44ab-ad0d-6ec755a45506 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:00.756462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3e67c-dad1-4f14-952a-9983f1338b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:00.757090 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3e67c-dad1-4f14-952a-9983f1338b3d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:00.757373 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3e67c-dad1-4f14-952a-9983f1338b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:00.757649 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c3e67c-dad1-4f14-952a-9983f1338b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:00.758303 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3e67c-dad1-4f14-952a-9983f1338b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:00.759965 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c3e67c-dad1-4f14-952a-9983f1338b3d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:00.771893 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1267/2543] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:11:00.776156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df40ee-9e31-4cd6-b341-8839b2a9354a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:00.787448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df40ee-9e31-4cd6-b341-8839b2a9354a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:00.792449 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d236e582-3de1-4d4c-895e-ee7e85e32f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:00.801828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236e582-3de1-4d4c-895e-ee7e85e32f05 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:00.801828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236e582-3de1-4d4c-895e-ee7e85e32f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:00.801828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d236e582-3de1-4d4c-895e-ee7e85e32f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:00.801828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236e582-3de1-4d4c-895e-ee7e85e32f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:00.801828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d236e582-3de1-4d4c-895e-ee7e85e32f05 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:00.808052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1277/2544] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:11:00.812380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3e67c-dad1-4f14-952a-9983f1338b3d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:00.826179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c3e67c-dad1-4f14-952a-9983f1338b3d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:00.832754 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcfaf0-e83c-45b2-bb5f-87ce9be9647c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:00.833439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcfaf0-e83c-45b2-bb5f-87ce9be9647c admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:00.833723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcfaf0-e83c-45b2-bb5f-87ce9be9647c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:00.834019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efcfaf0-e83c-45b2-bb5f-87ce9be9647c admin admin] PATH_INFO: `/v3/projects/131f86541d394d22804699fa37ec6b35/users/4b6cc8105d9f42f1a149d9383790b079/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:00.852538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcfaf0-e83c-45b2-bb5f-87ce9be9647c admin admin] RBAC: Authorizing `identity:create_grant(project_id=131f86541d394d22804699fa37ec6b35, user_id=4b6cc8105d9f42f1a149d9383790b079, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:00.855401 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efcfaf0-e83c-45b2-bb5f-87ce9be9647c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:00.976281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1268/2545] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:11:00 2022] PUT /identity/v3/projects/131f86541d394d22804699fa37ec6b35/users/4b6cc8105d9f42f1a149d9383790b079/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:00.988946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90dd38-66e8-47d8-a6f1-05e517a42746 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:00.989288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90dd38-66e8-47d8-a6f1-05e517a42746 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:00.989655 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90dd38-66e8-47d8-a6f1-05e517a42746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:01.030031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fe90dd38-66e8-47d8-a6f1-05e517a42746 None None] MFA Rules not processed for user `4b6cc8105d9f42f1a149d9383790b079`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:01.031248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fe90dd38-66e8-47d8-a6f1-05e517a42746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:01.102940 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1278/2546] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:11:01.114277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcfaf0-e83c-45b2-bb5f-87ce9be9647c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:01.116220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7efcfaf0-e83c-45b2-bb5f-87ce9be9647c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:01.174398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efcfaf0-e83c-45b2-bb5f-87ce9be9647c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:01.178226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146feaf-70a0-4ff8-b2c1-ee7a9b8995fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:01.178858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146feaf-70a0-4ff8-b2c1-ee7a9b8995fc service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:01.179134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146feaf-70a0-4ff8-b2c1-ee7a9b8995fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:01.179428 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146feaf-70a0-4ff8-b2c1-ee7a9b8995fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:01.188337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f146feaf-70a0-4ff8-b2c1-ee7a9b8995fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:01.189814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f146feaf-70a0-4ff8-b2c1-ee7a9b8995fc service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:01.198543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1269/2547] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:11:11.100033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:11.194588 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=50,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:11.494052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90dd38-66e8-47d8-a6f1-05e517a42746 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:11.611178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90dd38-66e8-47d8-a6f1-05e517a42746 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:11.615820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4ad5f-b6d9-499a-bc56-96a10362821e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:11.616457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4ad5f-b6d9-499a-bc56-96a10362821e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:11.616741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4ad5f-b6d9-499a-bc56-96a10362821e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:11.617018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b4ad5f-b6d9-499a-bc56-96a10362821e admin admin] PATH_INFO: `/v3/users/4b6cc8105d9f42f1a149d9383790b079` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:11.618215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4ad5f-b6d9-499a-bc56-96a10362821e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b6cc8105d9f42f1a149d9383790b079)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:11.620168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b4ad5f-b6d9-499a-bc56-96a10362821e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:11.823470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-05b4ad5f-b6d9-499a-bc56-96a10362821e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b6cc8105d9f42f1a149d9383790b079'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:11.910754 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-05b4ad5f-b6d9-499a-bc56-96a10362821e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b6cc8105d9f42f1a149d9383790b079'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:11.911381 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-05b4ad5f-b6d9-499a-bc56-96a10362821e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b6cc8105d9f42f1a149d9383790b079'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:11.912097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-05b4ad5f-b6d9-499a-bc56-96a10362821e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b6cc8105d9f42f1a149d9383790b079'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:11.929057 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-05b4ad5f-b6d9-499a-bc56-96a10362821e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b6cc8105d9f42f1a149d9383790b079'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:11.942476 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1279/2548] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:11:11 2022] DELETE /identity/v3/users/4b6cc8105d9f42f1a149d9383790b079 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:11.947093 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd84595-e1dd-4e0f-9193-7df5a7d277f9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:11.947381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd84595-e1dd-4e0f-9193-7df5a7d277f9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:11.947674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd84595-e1dd-4e0f-9193-7df5a7d277f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:12.011974 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0dd84595-e1dd-4e0f-9193-7df5a7d277f9 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:12.011974 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0dd84595-e1dd-4e0f-9193-7df5a7d277f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:12.071988 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1270/2549] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:12.082241 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4ad5f-b6d9-499a-bc56-96a10362821e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:12.083672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-05b4ad5f-b6d9-499a-bc56-96a10362821e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:12.128930 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:12.165583 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b4ad5f-b6d9-499a-bc56-96a10362821e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:12.179874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef50224-0417-4493-b06a-09bc73378e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:12.179874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef50224-0417-4493-b06a-09bc73378e52 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:12.179874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef50224-0417-4493-b06a-09bc73378e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:12.179874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef50224-0417-4493-b06a-09bc73378e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:12.181009 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef50224-0417-4493-b06a-09bc73378e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:12.182462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef50224-0417-4493-b06a-09bc73378e52 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:12.193508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1280/2550] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:11:12.681693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd84595-e1dd-4e0f-9193-7df5a7d277f9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:12.683306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0dd84595-e1dd-4e0f-9193-7df5a7d277f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:12.699194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd84595-e1dd-4e0f-9193-7df5a7d277f9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:12.707505 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfd533b-d46a-4a10-8696-196ca3b681a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:12.708602 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd533b-d46a-4a10-8696-196ca3b681a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:12.709052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd533b-d46a-4a10-8696-196ca3b681a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:12.709482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfd533b-d46a-4a10-8696-196ca3b681a5 admin admin] PATH_INFO: `/v3/projects/131f86541d394d22804699fa37ec6b35` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:12.711527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfd533b-d46a-4a10-8696-196ca3b681a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=131f86541d394d22804699fa37ec6b35)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:12.714817 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfd533b-d46a-4a10-8696-196ca3b681a5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:12.829078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2bfd533b-d46a-4a10-8696-196ca3b681a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '131f86541d394d22804699fa37ec6b35', 'request_id': 'req-2bfd533b-d46a-4a10-8696-196ca3b681a5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:12.836772 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2bfd533b-d46a-4a10-8696-196ca3b681a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '131f86541d394d22804699fa37ec6b35', 'request_id': 'req-2bfd533b-d46a-4a10-8696-196ca3b681a5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:12.905548 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2bfd533b-d46a-4a10-8696-196ca3b681a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '131f86541d394d22804699fa37ec6b35', 'request_id': 'req-2bfd533b-d46a-4a10-8696-196ca3b681a5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:12.920840 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2bfd533b-d46a-4a10-8696-196ca3b681a5 admin admin] The token cache is being invalidate because project 131f86541d394d22804699fa37ec6b35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:11:12.921375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2bfd533b-d46a-4a10-8696-196ca3b681a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:12.927984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2bfd533b-d46a-4a10-8696-196ca3b681a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:12.927984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1271/2551] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:11:12 2022] DELETE /identity/v3/projects/131f86541d394d22804699fa37ec6b35 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:12.930436 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221369e-f2eb-460b-889d-4c75a72e0d83 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:12.930756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221369e-f2eb-460b-889d-4c75a72e0d83 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:12.931071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221369e-f2eb-460b-889d-4c75a72e0d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:12.994171 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6221369e-f2eb-460b-889d-4c75a72e0d83 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:12.995431 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6221369e-f2eb-460b-889d-4c75a72e0d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:13.068841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1281/2552] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:13.077243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fc328-75ce-4d6a-bfe9-ae69190d88cd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:13.077551 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fc328-75ce-4d6a-bfe9-ae69190d88cd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:13.077825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178fc328-75ce-4d6a-bfe9-ae69190d88cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:13.099688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-178fc328-75ce-4d6a-bfe9-ae69190d88cd None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:13.100907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-178fc328-75ce-4d6a-bfe9-ae69190d88cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:13.108921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1272/2553] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:13.113582 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221369e-f2eb-460b-889d-4c75a72e0d83 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:13.123308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221369e-f2eb-460b-889d-4c75a72e0d83 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:13.128750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858dba4-a65c-4872-a8cc-daa687da745e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:13.129363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858dba4-a65c-4872-a8cc-daa687da745e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:13.129912 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858dba4-a65c-4872-a8cc-daa687da745e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:13.129912 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a858dba4-a65c-4872-a8cc-daa687da745e admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:13.130566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858dba4-a65c-4872-a8cc-daa687da745e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:11:13.130871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858dba4-a65c-4872-a8cc-daa687da745e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:13.132507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a858dba4-a65c-4872-a8cc-daa687da745e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:13.145243 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1282/2554] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:11:13.150829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fc328-75ce-4d6a-bfe9-ae69190d88cd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:13.161492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178fc328-75ce-4d6a-bfe9-ae69190d88cd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:13.172137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052eaa1d-95be-44d4-8369-3ad1b4599b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:13.172915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052eaa1d-95be-44d4-8369-3ad1b4599b70 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:13.173206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052eaa1d-95be-44d4-8369-3ad1b4599b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:13.174297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052eaa1d-95be-44d4-8369-3ad1b4599b70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:13.174297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052eaa1d-95be-44d4-8369-3ad1b4599b70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:13.176154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052eaa1d-95be-44d4-8369-3ad1b4599b70 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:13.274877 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1273/2555] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:11:13 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:13.279491 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858dba4-a65c-4872-a8cc-daa687da745e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:13.289878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a858dba4-a65c-4872-a8cc-daa687da745e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:13.351659 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac422c-157b-4181-b8a8-405d13c7d7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:13.352392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac422c-157b-4181-b8a8-405d13c7d7fe admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:13.352737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac422c-157b-4181-b8a8-405d13c7d7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:13.353019 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dac422c-157b-4181-b8a8-405d13c7d7fe admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:13.353658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac422c-157b-4181-b8a8-405d13c7d7fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:13.355611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dac422c-157b-4181-b8a8-405d13c7d7fe admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:13.449760 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1283/2556] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:11:13 2022] POST /identity/v3/users => generated 287 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:13.455332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052eaa1d-95be-44d4-8369-3ad1b4599b70 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:13.466929 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052eaa1d-95be-44d4-8369-3ad1b4599b70 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:13.472028 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c51d89-e537-4752-a125-3dd45dfb3906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:13.472640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c51d89-e537-4752-a125-3dd45dfb3906 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:13.473052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c51d89-e537-4752-a125-3dd45dfb3906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:13.473331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c51d89-e537-4752-a125-3dd45dfb3906 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:13.473982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c51d89-e537-4752-a125-3dd45dfb3906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:13.475595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c51d89-e537-4752-a125-3dd45dfb3906 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:13.487696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1274/2557] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:11:13.491919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac422c-157b-4181-b8a8-405d13c7d7fe admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:13.507082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dac422c-157b-4181-b8a8-405d13c7d7fe admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:13.512044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c33ef-492e-42b0-8bc4-7aa80ad0344d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:13.512665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c33ef-492e-42b0-8bc4-7aa80ad0344d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:13.512939 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c33ef-492e-42b0-8bc4-7aa80ad0344d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:13.513213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c33ef-492e-42b0-8bc4-7aa80ad0344d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:13.513878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c33ef-492e-42b0-8bc4-7aa80ad0344d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:13.515543 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c33ef-492e-42b0-8bc4-7aa80ad0344d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:13.530176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1284/2558] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:11:13.531321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c51d89-e537-4752-a125-3dd45dfb3906 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:13.541776 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c51d89-e537-4752-a125-3dd45dfb3906 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:13.546427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b16435-86c3-4c29-9079-5ab8ceeab4de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:13.547635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b16435-86c3-4c29-9079-5ab8ceeab4de admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:13.547991 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b16435-86c3-4c29-9079-5ab8ceeab4de admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:13.548271 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b16435-86c3-4c29-9079-5ab8ceeab4de admin admin] PATH_INFO: `/v3/projects/93a03c8359934981b364ce15d83e4284/users/151c7df28f60413386d222ec8817fe30/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:13.563790 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b16435-86c3-4c29-9079-5ab8ceeab4de admin admin] RBAC: Authorizing `identity:create_grant(project_id=93a03c8359934981b364ce15d83e4284, user_id=151c7df28f60413386d222ec8817fe30, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:13.566449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b16435-86c3-4c29-9079-5ab8ceeab4de admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:13.568842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:13.632783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1275/2559] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:11:13 2022] PUT /identity/v3/projects/93a03c8359934981b364ce15d83e4284/users/151c7df28f60413386d222ec8817fe30/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:13.644020 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5b784-2348-4edf-b8b5-a9ca527d8e7c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:13.644301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5b784-2348-4edf-b8b5-a9ca527d8e7c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:13.644570 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e5b784-2348-4edf-b8b5-a9ca527d8e7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:13.658003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:13.675170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-03e5b784-2348-4edf-b8b5-a9ca527d8e7c None None] MFA Rules not processed for user `151c7df28f60413386d222ec8817fe30`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:13.676510 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-03e5b784-2348-4edf-b8b5-a9ca527d8e7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:13.741330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1285/2560] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:11:13.748941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b16435-86c3-4c29-9079-5ab8ceeab4de admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:13.813966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b16435-86c3-4c29-9079-5ab8ceeab4de admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:13.818593 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29e9e8-998a-41d6-8e1e-5d836825e11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:13.819194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29e9e8-998a-41d6-8e1e-5d836825e11a admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:13.819470 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29e9e8-998a-41d6-8e1e-5d836825e11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:13.819773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f29e9e8-998a-41d6-8e1e-5d836825e11a admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:13.820409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29e9e8-998a-41d6-8e1e-5d836825e11a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:13.822817 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f29e9e8-998a-41d6-8e1e-5d836825e11a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:13.918452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1276/2561] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:11:13 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:13.923294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5b784-2348-4edf-b8b5-a9ca527d8e7c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:13.934091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e5b784-2348-4edf-b8b5-a9ca527d8e7c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:13.995591 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2dd330-5242-427d-ab4b-4a9fcf2e86f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:13.995591 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2dd330-5242-427d-ab4b-4a9fcf2e86f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:13.995591 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2dd330-5242-427d-ab4b-4a9fcf2e86f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:13.995591 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2dd330-5242-427d-ab4b-4a9fcf2e86f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:13.996098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2dd330-5242-427d-ab4b-4a9fcf2e86f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:13.997271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2dd330-5242-427d-ab4b-4a9fcf2e86f5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:14.118399 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1286/2562] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:11:13 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:14.122401 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29e9e8-998a-41d6-8e1e-5d836825e11a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:14.133447 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f29e9e8-998a-41d6-8e1e-5d836825e11a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:14.138071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a2466-02b7-497f-84fd-75972be873ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:14.138711 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a2466-02b7-497f-84fd-75972be873ad admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:14.138990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a2466-02b7-497f-84fd-75972be873ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:14.139261 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a2466-02b7-497f-84fd-75972be873ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:14.139930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a2466-02b7-497f-84fd-75972be873ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:14.141842 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a2466-02b7-497f-84fd-75972be873ad admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:14.153792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1277/2563] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:11:14.158603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2dd330-5242-427d-ab4b-4a9fcf2e86f5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:14.169043 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2dd330-5242-427d-ab4b-4a9fcf2e86f5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:14.174273 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640141b-940c-4df6-91ac-80e0d98b1897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:14.174925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640141b-940c-4df6-91ac-80e0d98b1897 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:14.175211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640141b-940c-4df6-91ac-80e0d98b1897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:14.175483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8640141b-940c-4df6-91ac-80e0d98b1897 admin admin] PATH_INFO: `/v3/projects/e6780ff04c314a6bac9a0fdba10fced9/users/df270e8f1a444400855c34aa9cc8d92f/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:14.190828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640141b-940c-4df6-91ac-80e0d98b1897 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6780ff04c314a6bac9a0fdba10fced9, user_id=df270e8f1a444400855c34aa9cc8d92f, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:14.193508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8640141b-940c-4df6-91ac-80e0d98b1897 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:14.266810 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1287/2564] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:11:14 2022] PUT /identity/v3/projects/e6780ff04c314a6bac9a0fdba10fced9/users/df270e8f1a444400855c34aa9cc8d92f/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:14.271302 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a2466-02b7-497f-84fd-75972be873ad admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:14.281729 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a2466-02b7-497f-84fd-75972be873ad admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:14.345364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c2f830-65e2-4c63-b884-c1431d70c938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:14.346016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2f830-65e2-4c63-b884-c1431d70c938 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:14.346235 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2f830-65e2-4c63-b884-c1431d70c938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:14.346504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c2f830-65e2-4c63-b884-c1431d70c938 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:14.347497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2f830-65e2-4c63-b884-c1431d70c938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:14.348772 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c2f830-65e2-4c63-b884-c1431d70c938 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:14.361154 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1278/2565] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:14 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:11:14.363977 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640141b-940c-4df6-91ac-80e0d98b1897 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:14.375470 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8640141b-940c-4df6-91ac-80e0d98b1897 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:14.380865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00ac63-0131-4b7f-8fa3-e464f88e3c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:14.381627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00ac63-0131-4b7f-8fa3-e464f88e3c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:14.381983 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00ac63-0131-4b7f-8fa3-e464f88e3c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:14.382333 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d00ac63-0131-4b7f-8fa3-e464f88e3c4e admin admin] PATH_INFO: `/v3/projects/e6780ff04c314a6bac9a0fdba10fced9/users/df270e8f1a444400855c34aa9cc8d92f/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:14.384522 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00ac63-0131-4b7f-8fa3-e464f88e3c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6780ff04c314a6bac9a0fdba10fced9, user_id=df270e8f1a444400855c34aa9cc8d92f, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:14.387286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d00ac63-0131-4b7f-8fa3-e464f88e3c4e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:14.412268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1288/2566] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:11:14 2022] PUT /identity/v3/projects/e6780ff04c314a6bac9a0fdba10fced9/users/df270e8f1a444400855c34aa9cc8d92f/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:14.413786 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5a6d1-20c5-4a38-bd5e-100aaf6bea57 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:14.414153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5a6d1-20c5-4a38-bd5e-100aaf6bea57 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:14.414530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5a6d1-20c5-4a38-bd5e-100aaf6bea57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:14.443770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-37e5a6d1-20c5-4a38-bd5e-100aaf6bea57 None None] MFA Rules not processed for user `df270e8f1a444400855c34aa9cc8d92f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:14.445009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-37e5a6d1-20c5-4a38-bd5e-100aaf6bea57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:14.507045 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:14.528346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1279/2567] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:14.541023 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00ac63-0131-4b7f-8fa3-e464f88e3c4e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:14.542730 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9d00ac63-0131-4b7f-8fa3-e464f88e3c4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:14.611323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:14.616219 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d00ac63-0131-4b7f-8fa3-e464f88e3c4e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:14.620141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ecc4b-440f-4612-9073-11f97ca0e8da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:14.620767 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ecc4b-440f-4612-9073-11f97ca0e8da service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:14.621057 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ecc4b-440f-4612-9073-11f97ca0e8da service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:14.621337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1ecc4b-440f-4612-9073-11f97ca0e8da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:14.632509 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ecc4b-440f-4612-9073-11f97ca0e8da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:14.634384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1ecc4b-440f-4612-9073-11f97ca0e8da service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:14.667729 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1289/2568] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:11:14.827431 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc66e88-1051-49c7-8cae-825720a42b37 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:14.827978 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc66e88-1051-49c7-8cae-825720a42b37 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:14.827978 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc66e88-1051-49c7-8cae-825720a42b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:14.858157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3fc66e88-1051-49c7-8cae-825720a42b37 None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:14.859420 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3fc66e88-1051-49c7-8cae-825720a42b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:14.932887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1280/2569] 10.222.0.67 () {58 vars in 1009 bytes} [Mon Sep 26 18:11:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:15.080676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ecc4b-440f-4612-9073-11f97ca0e8da service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:15.080676 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ff1ecc4b-440f-4612-9073-11f97ca0e8da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:15.167002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1ecc4b-440f-4612-9073-11f97ca0e8da service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:15.172327 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5318abb-9516-4b9a-8fe2-b84208763b63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:15.173116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5318abb-9516-4b9a-8fe2-b84208763b63 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:15.173350 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5318abb-9516-4b9a-8fe2-b84208763b63 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:15.173634 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5318abb-9516-4b9a-8fe2-b84208763b63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:15.182431 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5318abb-9516-4b9a-8fe2-b84208763b63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:15.184059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5318abb-9516-4b9a-8fe2-b84208763b63 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:15.192663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1290/2570] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:11:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:11:15.745356 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc66e88-1051-49c7-8cae-825720a42b37 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:15.880255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc66e88-1051-49c7-8cae-825720a42b37 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:15.887193 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f2831-7541-43d6-9bcc-b6e2e1ec03d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:15.888474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f2831-7541-43d6-9bcc-b6e2e1ec03d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:15.888757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f2831-7541-43d6-9bcc-b6e2e1ec03d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:15.889036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298f2831-7541-43d6-9bcc-b6e2e1ec03d7 admin admin] PATH_INFO: `/v3/users/151c7df28f60413386d222ec8817fe30` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:15.890958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f2831-7541-43d6-9bcc-b6e2e1ec03d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=151c7df28f60413386d222ec8817fe30)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:15.894255 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298f2831-7541-43d6-9bcc-b6e2e1ec03d7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:16.115258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-298f2831-7541-43d6-9bcc-b6e2e1ec03d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '151c7df28f60413386d222ec8817fe30'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:16.207109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-298f2831-7541-43d6-9bcc-b6e2e1ec03d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '151c7df28f60413386d222ec8817fe30'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:16.208332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-298f2831-7541-43d6-9bcc-b6e2e1ec03d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '151c7df28f60413386d222ec8817fe30'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:16.209303 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-298f2831-7541-43d6-9bcc-b6e2e1ec03d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '151c7df28f60413386d222ec8817fe30'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:16.225905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-298f2831-7541-43d6-9bcc-b6e2e1ec03d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '151c7df28f60413386d222ec8817fe30'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:16.251692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:16.260771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1281/2571] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:11:15 2022] DELETE /identity/v3/users/151c7df28f60413386d222ec8817fe30 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:16.264883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d08b1-128e-4c75-98fa-96af3c07add0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:16.269203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d08b1-128e-4c75-98fa-96af3c07add0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:16.269203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91d08b1-128e-4c75-98fa-96af3c07add0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:16.296498 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f91d08b1-128e-4c75-98fa-96af3c07add0 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:16.298545 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f91d08b1-128e-4c75-98fa-96af3c07add0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:16.401173 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1291/2572] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:16.411520 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f2831-7541-43d6-9bcc-b6e2e1ec03d7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:16.413092 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-298f2831-7541-43d6-9bcc-b6e2e1ec03d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:16.492829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298f2831-7541-43d6-9bcc-b6e2e1ec03d7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:16.496816 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb1c43-4edb-449f-b3bd-120765831e68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:16.497875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb1c43-4edb-449f-b3bd-120765831e68 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:16.498412 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb1c43-4edb-449f-b3bd-120765831e68 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:16.499035 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb1c43-4edb-449f-b3bd-120765831e68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:16.511451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb1c43-4edb-449f-b3bd-120765831e68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:16.513428 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb1c43-4edb-449f-b3bd-120765831e68 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:16.524757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1282/2573] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:11:17.169829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d08b1-128e-4c75-98fa-96af3c07add0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:17.171780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f91d08b1-128e-4c75-98fa-96af3c07add0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:17.239988 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91d08b1-128e-4c75-98fa-96af3c07add0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:17.254938 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da36870-ae0f-465a-9031-f8b91f884530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:17.256364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da36870-ae0f-465a-9031-f8b91f884530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:17.257354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da36870-ae0f-465a-9031-f8b91f884530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:17.257821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da36870-ae0f-465a-9031-f8b91f884530 admin admin] PATH_INFO: `/v3/projects/93a03c8359934981b364ce15d83e4284` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:17.262936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da36870-ae0f-465a-9031-f8b91f884530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93a03c8359934981b364ce15d83e4284)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:17.270231 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da36870-ae0f-465a-9031-f8b91f884530 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:17.459121 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2da36870-ae0f-465a-9031-f8b91f884530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93a03c8359934981b364ce15d83e4284', 'request_id': 'req-2da36870-ae0f-465a-9031-f8b91f884530'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:17.471079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2da36870-ae0f-465a-9031-f8b91f884530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93a03c8359934981b364ce15d83e4284', 'request_id': 'req-2da36870-ae0f-465a-9031-f8b91f884530'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:17.578912 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2da36870-ae0f-465a-9031-f8b91f884530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93a03c8359934981b364ce15d83e4284', 'request_id': 'req-2da36870-ae0f-465a-9031-f8b91f884530'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:17.597035 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2da36870-ae0f-465a-9031-f8b91f884530 admin admin] The token cache is being invalidate because project 93a03c8359934981b364ce15d83e4284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:11:17.597443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2da36870-ae0f-465a-9031-f8b91f884530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:17.603921 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2da36870-ae0f-465a-9031-f8b91f884530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:17.606015 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1292/2574] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:11:17 2022] DELETE /identity/v3/projects/93a03c8359934981b364ce15d83e4284 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:17.610496 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb1c43-4edb-449f-b3bd-120765831e68 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:17.623755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb1c43-4edb-449f-b3bd-120765831e68 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:17.707144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0037b44b-fdfa-40cf-a904-54b9a8b21a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:17.707735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037b44b-fdfa-40cf-a904-54b9a8b21a0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:17.708025 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037b44b-fdfa-40cf-a904-54b9a8b21a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:17.708301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0037b44b-fdfa-40cf-a904-54b9a8b21a0e admin admin] PATH_INFO: `/v3/users/df270e8f1a444400855c34aa9cc8d92f` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:17.709452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037b44b-fdfa-40cf-a904-54b9a8b21a0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=df270e8f1a444400855c34aa9cc8d92f)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:17.711472 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0037b44b-fdfa-40cf-a904-54b9a8b21a0e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:17.930386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0037b44b-fdfa-40cf-a904-54b9a8b21a0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df270e8f1a444400855c34aa9cc8d92f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:18.054799 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0037b44b-fdfa-40cf-a904-54b9a8b21a0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df270e8f1a444400855c34aa9cc8d92f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:18.059075 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0037b44b-fdfa-40cf-a904-54b9a8b21a0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df270e8f1a444400855c34aa9cc8d92f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:18.059075 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0037b44b-fdfa-40cf-a904-54b9a8b21a0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df270e8f1a444400855c34aa9cc8d92f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:18.067252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0037b44b-fdfa-40cf-a904-54b9a8b21a0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df270e8f1a444400855c34aa9cc8d92f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:18.094780 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1283/2575] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:11:17 2022] DELETE /identity/v3/users/df270e8f1a444400855c34aa9cc8d92f => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:18.955668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da36870-ae0f-465a-9031-f8b91f884530 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:18.957260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2da36870-ae0f-465a-9031-f8b91f884530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:18.966566 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:18.973867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da36870-ae0f-465a-9031-f8b91f884530 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:19.054398 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea51ee4-4f00-458c-81f8-cd34bd5f9c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:19.055058 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea51ee4-4f00-458c-81f8-cd34bd5f9c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:19.055913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea51ee4-4f00-458c-81f8-cd34bd5f9c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:19.072893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea51ee4-4f00-458c-81f8-cd34bd5f9c31 admin admin] PATH_INFO: `/v3/projects/e6780ff04c314a6bac9a0fdba10fced9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:19.072893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea51ee4-4f00-458c-81f8-cd34bd5f9c31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6780ff04c314a6bac9a0fdba10fced9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:19.072893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea51ee4-4f00-458c-81f8-cd34bd5f9c31 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:19.191717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dea51ee4-4f00-458c-81f8-cd34bd5f9c31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6780ff04c314a6bac9a0fdba10fced9', 'request_id': 'req-dea51ee4-4f00-458c-81f8-cd34bd5f9c31'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:19.201660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dea51ee4-4f00-458c-81f8-cd34bd5f9c31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6780ff04c314a6bac9a0fdba10fced9', 'request_id': 'req-dea51ee4-4f00-458c-81f8-cd34bd5f9c31'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:19.260925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dea51ee4-4f00-458c-81f8-cd34bd5f9c31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6780ff04c314a6bac9a0fdba10fced9', 'request_id': 'req-dea51ee4-4f00-458c-81f8-cd34bd5f9c31'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:19.275405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dea51ee4-4f00-458c-81f8-cd34bd5f9c31 admin admin] The token cache is being invalidate because project e6780ff04c314a6bac9a0fdba10fced9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:11:19.277717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dea51ee4-4f00-458c-81f8-cd34bd5f9c31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:19.277717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dea51ee4-4f00-458c-81f8-cd34bd5f9c31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:19.278554 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1293/2576] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:11:18 2022] DELETE /identity/v3/projects/e6780ff04c314a6bac9a0fdba10fced9 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:19.286353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8920783-250b-4443-afb1-0080e371847d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:19.286882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8920783-250b-4443-afb1-0080e371847d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:19.286882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8920783-250b-4443-afb1-0080e371847d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:19.313190 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c8920783-250b-4443-afb1-0080e371847d None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:19.314649 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c8920783-250b-4443-afb1-0080e371847d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:19.330197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:19.404266 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1284/2577] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:19.413062 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07e424-8319-4ec4-b6cb-3004e1774589 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:19.413325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07e424-8319-4ec4-b6cb-3004e1774589 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:19.413602 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca07e424-8319-4ec4-b6cb-3004e1774589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:19.436527 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ca07e424-8319-4ec4-b6cb-3004e1774589 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:19.438242 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ca07e424-8319-4ec4-b6cb-3004e1774589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:19.448969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1294/2578] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:19.454085 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8920783-250b-4443-afb1-0080e371847d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:19.464233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8920783-250b-4443-afb1-0080e371847d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:19.474439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3d620-a90a-454c-95d3-083f15eeffc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:19.475163 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3d620-a90a-454c-95d3-083f15eeffc9 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:19.475509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3d620-a90a-454c-95d3-083f15eeffc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:19.475886 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3d620-a90a-454c-95d3-083f15eeffc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:19.476661 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3d620-a90a-454c-95d3-083f15eeffc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:11:19.477029 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3d620-a90a-454c-95d3-083f15eeffc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:19.478983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3d620-a90a-454c-95d3-083f15eeffc9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:19.495838 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1285/2579] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:11:19.501559 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07e424-8319-4ec4-b6cb-3004e1774589 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:19.513907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca07e424-8319-4ec4-b6cb-3004e1774589 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:19.519609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904932e3-2de1-4cfd-948a-2c0995cbdb68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:19.520474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904932e3-2de1-4cfd-948a-2c0995cbdb68 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:19.520835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904932e3-2de1-4cfd-948a-2c0995cbdb68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:19.521193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904932e3-2de1-4cfd-948a-2c0995cbdb68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:19.521911 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904932e3-2de1-4cfd-948a-2c0995cbdb68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:19.524960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904932e3-2de1-4cfd-948a-2c0995cbdb68 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:19.652314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3d620-a90a-454c-95d3-083f15eeffc9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:19.659933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1295/2580] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:11:19 2022] POST /identity/v3/projects => generated 381 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:11:19.672900 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3d620-a90a-454c-95d3-083f15eeffc9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:19.795141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51602ec1-0976-4a7b-95be-5eede9827f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:19.796248 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51602ec1-0976-4a7b-95be-5eede9827f06 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:19.796679 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51602ec1-0976-4a7b-95be-5eede9827f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:19.797680 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51602ec1-0976-4a7b-95be-5eede9827f06 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:19.798410 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51602ec1-0976-4a7b-95be-5eede9827f06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:19.802304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51602ec1-0976-4a7b-95be-5eede9827f06 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:19.963029 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904932e3-2de1-4cfd-948a-2c0995cbdb68 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:19.973918 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1286/2581] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:11:19 2022] POST /identity/v3/users => generated 291 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:19.983991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904932e3-2de1-4cfd-948a-2c0995cbdb68 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:19.994161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e6a4f-08d8-4dd9-b2a2-d444472da5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:19.994913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e6a4f-08d8-4dd9-b2a2-d444472da5ba admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:19.995382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e6a4f-08d8-4dd9-b2a2-d444472da5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:19.995755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e6a4f-08d8-4dd9-b2a2-d444472da5ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:19.998884 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e6a4f-08d8-4dd9-b2a2-d444472da5ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:20.001260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340e6a4f-08d8-4dd9-b2a2-d444472da5ba admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:20.028067 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51602ec1-0976-4a7b-95be-5eede9827f06 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:20.040139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1296/2582] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:11:20.048291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51602ec1-0976-4a7b-95be-5eede9827f06 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:20.060480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254e9baf-1289-4f75-811a-ef0fc25a48c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:20.065391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e9baf-1289-4f75-811a-ef0fc25a48c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:20.065917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e9baf-1289-4f75-811a-ef0fc25a48c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:20.066359 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254e9baf-1289-4f75-811a-ef0fc25a48c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:20.072697 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e9baf-1289-4f75-811a-ef0fc25a48c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:20.074443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254e9baf-1289-4f75-811a-ef0fc25a48c3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:20.103389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1287/2583] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:11:20.105394 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e6a4f-08d8-4dd9-b2a2-d444472da5ba admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:20.118746 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e6a4f-08d8-4dd9-b2a2-d444472da5ba admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:20.124342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5232e-5d2d-41e4-8f17-b017f72929fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:20.125814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5232e-5d2d-41e4-8f17-b017f72929fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:20.126826 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5232e-5d2d-41e4-8f17-b017f72929fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:20.127474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f5232e-5d2d-41e4-8f17-b017f72929fc admin admin] PATH_INFO: `/v3/projects/cfb2520e7fb843618c787c80a0dff51e/users/b63840f8a2d84eff9acfd0f3db273264/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:20.161904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5232e-5d2d-41e4-8f17-b017f72929fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfb2520e7fb843618c787c80a0dff51e, user_id=b63840f8a2d84eff9acfd0f3db273264, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:20.172756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f5232e-5d2d-41e4-8f17-b017f72929fc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:20.279945 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1297/2584] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:11:20 2022] PUT /identity/v3/projects/cfb2520e7fb843618c787c80a0dff51e/users/b63840f8a2d84eff9acfd0f3db273264/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:20.283935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab719a-706b-40cb-82f5-2bb60bad8c01 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:20.284325 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab719a-706b-40cb-82f5-2bb60bad8c01 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:20.284527 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ab719a-706b-40cb-82f5-2bb60bad8c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:20.321652 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-92ab719a-706b-40cb-82f5-2bb60bad8c01 None None] MFA Rules not processed for user `b63840f8a2d84eff9acfd0f3db273264`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:20.322891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-92ab719a-706b-40cb-82f5-2bb60bad8c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:20.428887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1288/2585] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:20.433479 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5232e-5d2d-41e4-8f17-b017f72929fc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:20.493425 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:20.564297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f5232e-5d2d-41e4-8f17-b017f72929fc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:20.564297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10109f88-26bf-4881-862f-d8372fc38464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:20.564297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10109f88-26bf-4881-862f-d8372fc38464 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:20.564297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10109f88-26bf-4881-862f-d8372fc38464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:20.565326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10109f88-26bf-4881-862f-d8372fc38464 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:20.565326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10109f88-26bf-4881-862f-d8372fc38464 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:20.565326 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10109f88-26bf-4881-862f-d8372fc38464 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:20.687624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1298/2586] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:11:20 2022] POST /identity/v3/projects => generated 383 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:11:20.692669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab719a-706b-40cb-82f5-2bb60bad8c01 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:20.715933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ab719a-706b-40cb-82f5-2bb60bad8c01 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:20.739230 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:20.814599 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3d47c-06f8-4700-8619-e64ec2297066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:20.818555 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3d47c-06f8-4700-8619-e64ec2297066 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:20.819191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3d47c-06f8-4700-8619-e64ec2297066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:20.819735 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd3d47c-06f8-4700-8619-e64ec2297066 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:20.820655 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3d47c-06f8-4700-8619-e64ec2297066 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:20.824426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd3d47c-06f8-4700-8619-e64ec2297066 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:20.965812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1289/2587] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:11:20 2022] POST /identity/v3/users => generated 298 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:20.967126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10109f88-26bf-4881-862f-d8372fc38464 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:20.987208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10109f88-26bf-4881-862f-d8372fc38464 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:20.994177 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e88877-09a6-4ef4-ae2d-1a5442a445f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:20.995112 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e88877-09a6-4ef4-ae2d-1a5442a445f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:20.995297 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e88877-09a6-4ef4-ae2d-1a5442a445f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:20.995581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e88877-09a6-4ef4-ae2d-1a5442a445f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:20.996344 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e88877-09a6-4ef4-ae2d-1a5442a445f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:20.998313 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e88877-09a6-4ef4-ae2d-1a5442a445f9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:21.016292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1299/2588] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:11:21.021774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3d47c-06f8-4700-8619-e64ec2297066 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:21.043341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd3d47c-06f8-4700-8619-e64ec2297066 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:21.052193 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778e613-aa2d-4b12-bc25-adaedefc2c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:21.053530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778e613-aa2d-4b12-bc25-adaedefc2c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:21.054031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778e613-aa2d-4b12-bc25-adaedefc2c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:21.054577 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778e613-aa2d-4b12-bc25-adaedefc2c16 admin admin] PATH_INFO: `/v3/projects/6dd65d66f99b485aba2e45f1ae85d491/users/cd777c6709eb4c0e8be7291154a11508/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:21.088780 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778e613-aa2d-4b12-bc25-adaedefc2c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd65d66f99b485aba2e45f1ae85d491, user_id=cd777c6709eb4c0e8be7291154a11508, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:21.091406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1778e613-aa2d-4b12-bc25-adaedefc2c16 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:21.167427 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1290/2589] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:11:21 2022] PUT /identity/v3/projects/6dd65d66f99b485aba2e45f1ae85d491/users/cd777c6709eb4c0e8be7291154a11508/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:21.171556 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e88877-09a6-4ef4-ae2d-1a5442a445f9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:21.183260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e88877-09a6-4ef4-ae2d-1a5442a445f9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:21.290282 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350514c3-8d8f-4cbd-bf0d-68c1fa848ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:21.291374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350514c3-8d8f-4cbd-bf0d-68c1fa848ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:21.292008 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350514c3-8d8f-4cbd-bf0d-68c1fa848ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:21.292552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350514c3-8d8f-4cbd-bf0d-68c1fa848ca3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:21.293656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350514c3-8d8f-4cbd-bf0d-68c1fa848ca3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:21.300829 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350514c3-8d8f-4cbd-bf0d-68c1fa848ca3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:21.337448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1300/2590] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:21 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:11:21.338348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778e613-aa2d-4b12-bc25-adaedefc2c16 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:21.355087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778e613-aa2d-4b12-bc25-adaedefc2c16 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:21.363438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b1ed3-dc97-4ac5-a463-e32fa997049a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:21.364626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b1ed3-dc97-4ac5-a463-e32fa997049a admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:21.365219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b1ed3-dc97-4ac5-a463-e32fa997049a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:21.365668 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0b1ed3-dc97-4ac5-a463-e32fa997049a admin admin] PATH_INFO: `/v3/projects/6dd65d66f99b485aba2e45f1ae85d491/users/cd777c6709eb4c0e8be7291154a11508/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:21.370004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b1ed3-dc97-4ac5-a463-e32fa997049a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dd65d66f99b485aba2e45f1ae85d491, user_id=cd777c6709eb4c0e8be7291154a11508, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:21.374815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0b1ed3-dc97-4ac5-a463-e32fa997049a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:21.390408 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1291/2591] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:11:21 2022] PUT /identity/v3/projects/6dd65d66f99b485aba2e45f1ae85d491/users/cd777c6709eb4c0e8be7291154a11508/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:21.406121 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86166f-52fd-4e6e-a6e3-bdaa6f8bdcaa None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:21.406720 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86166f-52fd-4e6e-a6e3-bdaa6f8bdcaa None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:21.407166 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86166f-52fd-4e6e-a6e3-bdaa6f8bdcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:21.458211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8f86166f-52fd-4e6e-a6e3-bdaa6f8bdcaa None None] MFA Rules not processed for user `cd777c6709eb4c0e8be7291154a11508`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:21.459684 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8f86166f-52fd-4e6e-a6e3-bdaa6f8bdcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:21.614775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1301/2592] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:21.622841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b1ed3-dc97-4ac5-a463-e32fa997049a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:21.625970 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9e0b1ed3-dc97-4ac5-a463-e32fa997049a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:21.709317 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0b1ed3-dc97-4ac5-a463-e32fa997049a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:21.716969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652a2fc-3109-4b3b-8df6-2d37b6397aa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:21.716969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652a2fc-3109-4b3b-8df6-2d37b6397aa7 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:21.716969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652a2fc-3109-4b3b-8df6-2d37b6397aa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:21.716969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652a2fc-3109-4b3b-8df6-2d37b6397aa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:21.727383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652a2fc-3109-4b3b-8df6-2d37b6397aa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:21.729629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652a2fc-3109-4b3b-8df6-2d37b6397aa7 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:21.744262 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1292/2593] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:11:21.944969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86166f-52fd-4e6e-a6e3-bdaa6f8bdcaa None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:21.955739 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86166f-52fd-4e6e-a6e3-bdaa6f8bdcaa None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:21.967463 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3eef2-f689-412c-9c08-0917727c3b71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:21.970827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3eef2-f689-412c-9c08-0917727c3b71 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:21.970827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3eef2-f689-412c-9c08-0917727c3b71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:21.970827 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3eef2-f689-412c-9c08-0917727c3b71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:22.020481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:22.075667 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3eef2-f689-412c-9c08-0917727c3b71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:22.077605 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3eef2-f689-412c-9c08-0917727c3b71 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:22.098828 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1302/2594] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:21 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:11:23.019204 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652a2fc-3109-4b3b-8df6-2d37b6397aa7 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:23.087556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652a2fc-3109-4b3b-8df6-2d37b6397aa7 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:23.092375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca669d2-d1b8-4a2b-ba6c-2058a672e3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:23.094492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca669d2-d1b8-4a2b-ba6c-2058a672e3df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:23.094492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca669d2-d1b8-4a2b-ba6c-2058a672e3df admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:23.094492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca669d2-d1b8-4a2b-ba6c-2058a672e3df admin admin] PATH_INFO: `/v3/users/b63840f8a2d84eff9acfd0f3db273264` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:23.096087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca669d2-d1b8-4a2b-ba6c-2058a672e3df admin admin] RBAC: Authorizing `identity:delete_user(user_id=b63840f8a2d84eff9acfd0f3db273264)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:23.097651 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca669d2-d1b8-4a2b-ba6c-2058a672e3df admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:23.112216 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:23.392747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dca669d2-d1b8-4a2b-ba6c-2058a672e3df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b63840f8a2d84eff9acfd0f3db273264'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:23.477827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dca669d2-d1b8-4a2b-ba6c-2058a672e3df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b63840f8a2d84eff9acfd0f3db273264'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:23.478600 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dca669d2-d1b8-4a2b-ba6c-2058a672e3df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b63840f8a2d84eff9acfd0f3db273264'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:23.479289 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dca669d2-d1b8-4a2b-ba6c-2058a672e3df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b63840f8a2d84eff9acfd0f3db273264'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:23.488987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dca669d2-d1b8-4a2b-ba6c-2058a672e3df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b63840f8a2d84eff9acfd0f3db273264'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:23.513315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1293/2595] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:11:23 2022] DELETE /identity/v3/users/b63840f8a2d84eff9acfd0f3db273264 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:23.519869 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658f05b-bb15-4f64-97cd-c3af377a83a8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:23.520633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658f05b-bb15-4f64-97cd-c3af377a83a8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:23.523541 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2658f05b-bb15-4f64-97cd-c3af377a83a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:23.553514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2658f05b-bb15-4f64-97cd-c3af377a83a8 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:23.554851 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2658f05b-bb15-4f64-97cd-c3af377a83a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:23.660696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1303/2596] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:23.673085 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca669d2-d1b8-4a2b-ba6c-2058a672e3df admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:23.674713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dca669d2-d1b8-4a2b-ba6c-2058a672e3df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:23.758578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca669d2-d1b8-4a2b-ba6c-2058a672e3df admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:23.766739 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae81b9-1a51-47cb-ba10-08909687e41f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:23.767999 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae81b9-1a51-47cb-ba10-08909687e41f service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:23.768436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae81b9-1a51-47cb-ba10-08909687e41f service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:23.768861 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae81b9-1a51-47cb-ba10-08909687e41f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:23.787763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae81b9-1a51-47cb-ba10-08909687e41f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:23.788036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae81b9-1a51-47cb-ba10-08909687e41f service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:23.805274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1294/2597] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:11:24.469421 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658f05b-bb15-4f64-97cd-c3af377a83a8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:24.471320 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2658f05b-bb15-4f64-97cd-c3af377a83a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:24.486956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2658f05b-bb15-4f64-97cd-c3af377a83a8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:24.501082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4dda1-adf7-443a-89af-dd4b80d033f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:24.502093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4dda1-adf7-443a-89af-dd4b80d033f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:24.502467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4dda1-adf7-443a-89af-dd4b80d033f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:24.502807 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4dda1-adf7-443a-89af-dd4b80d033f4 admin admin] PATH_INFO: `/v3/projects/cfb2520e7fb843618c787c80a0dff51e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:24.504561 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4dda1-adf7-443a-89af-dd4b80d033f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfb2520e7fb843618c787c80a0dff51e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:24.506895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4dda1-adf7-443a-89af-dd4b80d033f4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:24.637599 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7dc4dda1-adf7-443a-89af-dd4b80d033f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfb2520e7fb843618c787c80a0dff51e', 'request_id': 'req-7dc4dda1-adf7-443a-89af-dd4b80d033f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:24.650617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7dc4dda1-adf7-443a-89af-dd4b80d033f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfb2520e7fb843618c787c80a0dff51e', 'request_id': 'req-7dc4dda1-adf7-443a-89af-dd4b80d033f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:24.740347 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7dc4dda1-adf7-443a-89af-dd4b80d033f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfb2520e7fb843618c787c80a0dff51e', 'request_id': 'req-7dc4dda1-adf7-443a-89af-dd4b80d033f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:24.767995 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7dc4dda1-adf7-443a-89af-dd4b80d033f4 admin admin] The token cache is being invalidate because project cfb2520e7fb843618c787c80a0dff51e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:11:24.768437 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7dc4dda1-adf7-443a-89af-dd4b80d033f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:24.769226 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7dc4dda1-adf7-443a-89af-dd4b80d033f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:24.771065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1304/2598] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:11:24 2022] DELETE /identity/v3/projects/cfb2520e7fb843618c787c80a0dff51e => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:24.775773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae81b9-1a51-47cb-ba10-08909687e41f service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:24.789306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae81b9-1a51-47cb-ba10-08909687e41f service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:24.893049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6407d8-78ea-46d2-a281-b819b9453e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:24.893689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6407d8-78ea-46d2-a281-b819b9453e9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:24.894043 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6407d8-78ea-46d2-a281-b819b9453e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:24.894314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6407d8-78ea-46d2-a281-b819b9453e9e admin admin] PATH_INFO: `/v3/users/cd777c6709eb4c0e8be7291154a11508` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:24.895559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6407d8-78ea-46d2-a281-b819b9453e9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd777c6709eb4c0e8be7291154a11508)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:24.899231 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6407d8-78ea-46d2-a281-b819b9453e9e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:25.161259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:25.180536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cd6407d8-78ea-46d2-a281-b819b9453e9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd777c6709eb4c0e8be7291154a11508'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:25.272389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cd6407d8-78ea-46d2-a281-b819b9453e9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd777c6709eb4c0e8be7291154a11508'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:25.273186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cd6407d8-78ea-46d2-a281-b819b9453e9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd777c6709eb4c0e8be7291154a11508'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:25.274405 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cd6407d8-78ea-46d2-a281-b819b9453e9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd777c6709eb4c0e8be7291154a11508'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:25.303917 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cd6407d8-78ea-46d2-a281-b819b9453e9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd777c6709eb4c0e8be7291154a11508'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:25.409433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1295/2599] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:11:24 2022] DELETE /identity/v3/users/cd777c6709eb4c0e8be7291154a11508 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:25.943622 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4dda1-adf7-443a-89af-dd4b80d033f4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:25.945931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7dc4dda1-adf7-443a-89af-dd4b80d033f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:25.977032 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4dda1-adf7-443a-89af-dd4b80d033f4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:26.040774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:26.058922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0067934-b63c-4699-b5e5-671c667570a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:26.059628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0067934-b63c-4699-b5e5-671c667570a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:26.059959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0067934-b63c-4699-b5e5-671c667570a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:26.060238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0067934-b63c-4699-b5e5-671c667570a2 admin admin] PATH_INFO: `/v3/projects/6dd65d66f99b485aba2e45f1ae85d491` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:26.061482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0067934-b63c-4699-b5e5-671c667570a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dd65d66f99b485aba2e45f1ae85d491)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:26.064842 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0067934-b63c-4699-b5e5-671c667570a2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:26.181709 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b0067934-b63c-4699-b5e5-671c667570a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dd65d66f99b485aba2e45f1ae85d491', 'request_id': 'req-b0067934-b63c-4699-b5e5-671c667570a2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:26.192277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b0067934-b63c-4699-b5e5-671c667570a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dd65d66f99b485aba2e45f1ae85d491', 'request_id': 'req-b0067934-b63c-4699-b5e5-671c667570a2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:26.260998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b0067934-b63c-4699-b5e5-671c667570a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dd65d66f99b485aba2e45f1ae85d491', 'request_id': 'req-b0067934-b63c-4699-b5e5-671c667570a2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:26.275573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b0067934-b63c-4699-b5e5-671c667570a2 admin admin] The token cache is being invalidate because project 6dd65d66f99b485aba2e45f1ae85d491 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:11:26.275965 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b0067934-b63c-4699-b5e5-671c667570a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:26.276688 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b0067934-b63c-4699-b5e5-671c667570a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:26.278493 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1305/2600] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:11:25 2022] DELETE /identity/v3/projects/6dd65d66f99b485aba2e45f1ae85d491 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:26.284361 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5e667-34d2-46e9-8533-007d1cc2bf62 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:26.284915 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5e667-34d2-46e9-8533-007d1cc2bf62 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:26.285045 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5e667-34d2-46e9-8533-007d1cc2bf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:26.308100 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-43d5e667-34d2-46e9-8533-007d1cc2bf62 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:26.309435 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-43d5e667-34d2-46e9-8533-007d1cc2bf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:26.383114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1296/2601] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:26.391990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5e7fb-22aa-4aa7-a483-be8d525552e8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:26.392378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5e7fb-22aa-4aa7-a483-be8d525552e8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:26.392986 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5e7fb-22aa-4aa7-a483-be8d525552e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:26.418240 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3be5e7fb-22aa-4aa7-a483-be8d525552e8 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:26.419498 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3be5e7fb-22aa-4aa7-a483-be8d525552e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:26.428172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1306/2602] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:26.432708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5e667-34d2-46e9-8533-007d1cc2bf62 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:26.443869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5e667-34d2-46e9-8533-007d1cc2bf62 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:26.448595 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecaf175-3b5d-41b6-998c-6289167e01d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:26.449331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecaf175-3b5d-41b6-998c-6289167e01d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:26.449615 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecaf175-3b5d-41b6-998c-6289167e01d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:26.449912 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ecaf175-3b5d-41b6-998c-6289167e01d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:26.450756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecaf175-3b5d-41b6-998c-6289167e01d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:11:26.451085 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecaf175-3b5d-41b6-998c-6289167e01d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:26.452793 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ecaf175-3b5d-41b6-998c-6289167e01d6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:26.465529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1297/2603] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:11:26.470253 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5e7fb-22aa-4aa7-a483-be8d525552e8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:26.480537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5e7fb-22aa-4aa7-a483-be8d525552e8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:26.486451 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f5acc-e313-4713-a646-fb5ec15c5a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:26.487059 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f5acc-e313-4713-a646-fb5ec15c5a4c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:26.487341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f5acc-e313-4713-a646-fb5ec15c5a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:26.487618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f5acc-e313-4713-a646-fb5ec15c5a4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:26.488286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f5acc-e313-4713-a646-fb5ec15c5a4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:26.490176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f5acc-e313-4713-a646-fb5ec15c5a4c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:26.737674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1307/2604] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:11:26 2022] POST /identity/v3/projects => generated 383 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:26.746973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecaf175-3b5d-41b6-998c-6289167e01d6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:26.750497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ecaf175-3b5d-41b6-998c-6289167e01d6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:26.827800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd9125-b74d-47d1-aa9b-3f263a4322c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:26.829014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd9125-b74d-47d1-aa9b-3f263a4322c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:26.829014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd9125-b74d-47d1-aa9b-3f263a4322c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:26.829434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd9125-b74d-47d1-aa9b-3f263a4322c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:26.829847 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd9125-b74d-47d1-aa9b-3f263a4322c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:26.831903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebd9125-b74d-47d1-aa9b-3f263a4322c6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:26.948394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1298/2605] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:11:26 2022] POST /identity/v3/users => generated 292 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:26.952528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f5acc-e313-4713-a646-fb5ec15c5a4c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:26.964259 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f5acc-e313-4713-a646-fb5ec15c5a4c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:26.970146 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162783f-3e0d-44be-9e06-bb331c7fe7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:26.976442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162783f-3e0d-44be-9e06-bb331c7fe7ed admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:26.976442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162783f-3e0d-44be-9e06-bb331c7fe7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:26.976442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162783f-3e0d-44be-9e06-bb331c7fe7ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:26.976442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162783f-3e0d-44be-9e06-bb331c7fe7ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:26.976442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162783f-3e0d-44be-9e06-bb331c7fe7ed admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:26.996536 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1308/2606] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:11:27.001130 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd9125-b74d-47d1-aa9b-3f263a4322c6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:27.011949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd9125-b74d-47d1-aa9b-3f263a4322c6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:27.018214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41da6680-dfb5-4410-82a0-82c6591a467c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:27.018958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da6680-dfb5-4410-82a0-82c6591a467c admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:27.019236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da6680-dfb5-4410-82a0-82c6591a467c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:27.019513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41da6680-dfb5-4410-82a0-82c6591a467c admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:27.020237 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da6680-dfb5-4410-82a0-82c6591a467c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:27.021839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41da6680-dfb5-4410-82a0-82c6591a467c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:27.034512 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1299/2607] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:11:27.038852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162783f-3e0d-44be-9e06-bb331c7fe7ed admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:27.049876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162783f-3e0d-44be-9e06-bb331c7fe7ed admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:27.054844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3121e4-4d0c-4255-a0e8-d59cb670e02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:27.055530 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3121e4-4d0c-4255-a0e8-d59cb670e02c admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:27.056188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3121e4-4d0c-4255-a0e8-d59cb670e02c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:27.056188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3121e4-4d0c-4255-a0e8-d59cb670e02c admin admin] PATH_INFO: `/v3/projects/574db4595c454f0fbd8672928b7ca212/users/614fdacf9d7e46c496beaa774c9c3b42/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:27.075613 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3121e4-4d0c-4255-a0e8-d59cb670e02c admin admin] RBAC: Authorizing `identity:create_grant(project_id=574db4595c454f0fbd8672928b7ca212, user_id=614fdacf9d7e46c496beaa774c9c3b42, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:27.078910 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3121e4-4d0c-4255-a0e8-d59cb670e02c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:27.146622 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1309/2608] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:11:27 2022] PUT /identity/v3/projects/574db4595c454f0fbd8672928b7ca212/users/614fdacf9d7e46c496beaa774c9c3b42/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:27.158898 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3601816-f536-4338-8816-25a21e0799f0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:27.159191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3601816-f536-4338-8816-25a21e0799f0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:27.159467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3601816-f536-4338-8816-25a21e0799f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:27.197452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c3601816-f536-4338-8816-25a21e0799f0 None None] MFA Rules not processed for user `614fdacf9d7e46c496beaa774c9c3b42`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:27.198859 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c3601816-f536-4338-8816-25a21e0799f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:27.205693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:27.303430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1300/2609] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:27.305460 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3121e4-4d0c-4255-a0e8-d59cb670e02c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:27.424638 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3121e4-4d0c-4255-a0e8-d59cb670e02c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:27.430044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43aee80-08c2-4704-b8c3-7e0125759517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:27.430855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43aee80-08c2-4704-b8c3-7e0125759517 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:27.431144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43aee80-08c2-4704-b8c3-7e0125759517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:27.431457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43aee80-08c2-4704-b8c3-7e0125759517 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:27.432167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43aee80-08c2-4704-b8c3-7e0125759517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:27.436625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43aee80-08c2-4704-b8c3-7e0125759517 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:27.554551 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:27.569257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1310/2610] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:11:27 2022] POST /identity/v3/projects => generated 383 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:11:27.577091 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3601816-f536-4338-8816-25a21e0799f0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:27.589426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3601816-f536-4338-8816-25a21e0799f0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:27.683851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e788fc-0ec4-495e-94e7-cae45b1ca15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:27.684614 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e788fc-0ec4-495e-94e7-cae45b1ca15d admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:27.685855 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e788fc-0ec4-495e-94e7-cae45b1ca15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:27.685855 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e788fc-0ec4-495e-94e7-cae45b1ca15d admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:27.686153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e788fc-0ec4-495e-94e7-cae45b1ca15d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:27.688556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e788fc-0ec4-495e-94e7-cae45b1ca15d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:27.845052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1301/2611] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:11:27 2022] POST /identity/v3/users => generated 298 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:27.849058 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43aee80-08c2-4704-b8c3-7e0125759517 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:27.861356 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43aee80-08c2-4704-b8c3-7e0125759517 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:27.868464 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf47cfe-6db6-4f3d-b0c6-cd211bcb1351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:27.869513 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf47cfe-6db6-4f3d-b0c6-cd211bcb1351 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:27.869978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf47cfe-6db6-4f3d-b0c6-cd211bcb1351 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:27.870659 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf47cfe-6db6-4f3d-b0c6-cd211bcb1351 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:27.871805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf47cfe-6db6-4f3d-b0c6-cd211bcb1351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:27.873968 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf47cfe-6db6-4f3d-b0c6-cd211bcb1351 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:27.886921 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1311/2612] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:11:27.894321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e788fc-0ec4-495e-94e7-cae45b1ca15d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:27.904807 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e788fc-0ec4-495e-94e7-cae45b1ca15d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:27.910360 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af292e-fd33-4605-b55b-3111a3144ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:27.911035 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af292e-fd33-4605-b55b-3111a3144ca8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:27.911316 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af292e-fd33-4605-b55b-3111a3144ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:27.911592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af292e-fd33-4605-b55b-3111a3144ca8 admin admin] PATH_INFO: `/v3/projects/7404835c03f0473d8657a3c979baa3e2/users/abae3fca6d734473bdeb0f763351272d/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:27.948923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af292e-fd33-4605-b55b-3111a3144ca8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7404835c03f0473d8657a3c979baa3e2, user_id=abae3fca6d734473bdeb0f763351272d, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:27.952183 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af292e-fd33-4605-b55b-3111a3144ca8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:28.083822 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1302/2613] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:11:27 2022] PUT /identity/v3/projects/7404835c03f0473d8657a3c979baa3e2/users/abae3fca6d734473bdeb0f763351272d/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:28.088482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf47cfe-6db6-4f3d-b0c6-cd211bcb1351 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:28.106357 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf47cfe-6db6-4f3d-b0c6-cd211bcb1351 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:28.172911 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1f0f8b-6997-4c76-8f58-2bb014b6a117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:28.173639 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f0f8b-6997-4c76-8f58-2bb014b6a117 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:28.174002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f0f8b-6997-4c76-8f58-2bb014b6a117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:28.174351 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1f0f8b-6997-4c76-8f58-2bb014b6a117 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:28.175078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f0f8b-6997-4c76-8f58-2bb014b6a117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:28.176818 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1f0f8b-6997-4c76-8f58-2bb014b6a117 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:28.189390 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1312/2614] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:28 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:11:28.193873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af292e-fd33-4605-b55b-3111a3144ca8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:28.206358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af292e-fd33-4605-b55b-3111a3144ca8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:28.222485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9f4a7-8bb6-4072-98b3-33209c744830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:28.223275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9f4a7-8bb6-4072-98b3-33209c744830 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:28.223692 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9f4a7-8bb6-4072-98b3-33209c744830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:28.224047 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9f4a7-8bb6-4072-98b3-33209c744830 admin admin] PATH_INFO: `/v3/projects/7404835c03f0473d8657a3c979baa3e2/users/abae3fca6d734473bdeb0f763351272d/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:28.226996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9f4a7-8bb6-4072-98b3-33209c744830 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7404835c03f0473d8657a3c979baa3e2, user_id=abae3fca6d734473bdeb0f763351272d, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:28.230324 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9f4a7-8bb6-4072-98b3-33209c744830 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:28.247473 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1303/2615] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:11:28 2022] PUT /identity/v3/projects/7404835c03f0473d8657a3c979baa3e2/users/abae3fca6d734473bdeb0f763351272d/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:11:28.259273 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d01eb-4149-47ae-806e-4e998adf7d1d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:28.259747 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d01eb-4149-47ae-806e-4e998adf7d1d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:28.263060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d01eb-4149-47ae-806e-4e998adf7d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:28.302178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-425d01eb-4149-47ae-806e-4e998adf7d1d None None] MFA Rules not processed for user `abae3fca6d734473bdeb0f763351272d`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:28.303594 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-425d01eb-4149-47ae-806e-4e998adf7d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:28.393411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1313/2616] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:28.404277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9f4a7-8bb6-4072-98b3-33209c744830 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:28.405983 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0de9f4a7-8bb6-4072-98b3-33209c744830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:28.465360 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9f4a7-8bb6-4072-98b3-33209c744830 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:28.469759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7963e8-f4c6-495c-8cde-5d1fad50f21d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:28.470483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7963e8-f4c6-495c-8cde-5d1fad50f21d service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:28.470764 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7963e8-f4c6-495c-8cde-5d1fad50f21d service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:28.471035 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7963e8-f4c6-495c-8cde-5d1fad50f21d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:28.483889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:28.532465 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7963e8-f4c6-495c-8cde-5d1fad50f21d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:28.534003 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7963e8-f4c6-495c-8cde-5d1fad50f21d service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:28.545672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1304/2617] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:11:34.197432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d01eb-4149-47ae-806e-4e998adf7d1d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:34.217516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d01eb-4149-47ae-806e-4e998adf7d1d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:34.221966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4222eee1-943f-4d0a-b453-a04d54c3777c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:34.222501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222eee1-943f-4d0a-b453-a04d54c3777c service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:34.229197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222eee1-943f-4d0a-b453-a04d54c3777c service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:34.229197 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4222eee1-943f-4d0a-b453-a04d54c3777c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:34.232769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222eee1-943f-4d0a-b453-a04d54c3777c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:34.234807 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4222eee1-943f-4d0a-b453-a04d54c3777c service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:34.245311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1314/2618] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:11:36.457583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0059d-68bc-4705-a96b-924967f1a2b1 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:36.458223 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0059d-68bc-4705-a96b-924967f1a2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:36.458404 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0059d-68bc-4705-a96b-924967f1a2b1 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:36.459912 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1305/2619] 10.222.0.67 () {54 vars in 921 bytes} [Mon Sep 26 18:11:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:11:36.466727 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbc64b-3cf9-4033-b69d-a14d88d76cb3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:36.467594 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbc64b-3cf9-4033-b69d-a14d88d76cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:36.467910 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adbc64b-3cf9-4033-b69d-a14d88d76cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:36.501950 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9adbc64b-3cf9-4033-b69d-a14d88d76cb3 None None] MFA Rules not processed for user `b9ed29a3f6be4d64bb3751f9e95eb56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:36.503210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9adbc64b-3cf9-4033-b69d-a14d88d76cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:36.548484 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:36.601086 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1315/2620] 10.222.0.67 () {58 vars in 1018 bytes} [Mon Sep 26 18:11:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:11:36.615748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0059d-68bc-4705-a96b-924967f1a2b1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:36.619123 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ffb0059d-68bc-4705-a96b-924967f1a2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:36.820654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0059d-68bc-4705-a96b-924967f1a2b1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:36.824589 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa91d525-14b1-4aa4-ac06-d512e9872a44 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6dee005fc3d4fb593a1771e7851c20a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:36.825192 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91d525-14b1-4aa4-ac06-d512e9872a44 service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:36.825501 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91d525-14b1-4aa4-ac06-d512e9872a44 service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:36.825811 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa91d525-14b1-4aa4-ac06-d512e9872a44 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:36.836969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91d525-14b1-4aa4-ac06-d512e9872a44 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:36.838409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa91d525-14b1-4aa4-ac06-d512e9872a44 service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:36.850678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1306/2621] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:11:37.273982 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194825c-acfd-4ff2-ad43-8e7243619bdd None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:37.274243 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194825c-acfd-4ff2-ad43-8e7243619bdd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:37.274633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2194825c-acfd-4ff2-ad43-8e7243619bdd None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:37.276227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1316/2622] 10.222.0.67 () {54 vars in 921 bytes} [Mon Sep 26 18:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:11:37.281662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88af1-5dda-4f58-a3c2-f1d445b88076 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:37.281916 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88af1-5dda-4f58-a3c2-f1d445b88076 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:37.282190 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef88af1-5dda-4f58-a3c2-f1d445b88076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:37.326843 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4ef88af1-5dda-4f58-a3c2-f1d445b88076 None None] MFA Rules not processed for user `ef7e0b772a284e7da4fae7d73f34d3de`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:37.333028 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4ef88af1-5dda-4f58-a3c2-f1d445b88076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:37.399347 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1307/2623] 10.222.0.67 () {58 vars in 1018 bytes} [Mon Sep 26 18:11:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:11:37.415208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194825c-acfd-4ff2-ad43-8e7243619bdd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:37.417266 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2194825c-acfd-4ff2-ad43-8e7243619bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:37.434142 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2194825c-acfd-4ff2-ad43-8e7243619bdd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:37.440912 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d2a4c6-2305-415c-a3bc-ac4b53eb0ff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:37.441937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2a4c6-2305-415c-a3bc-ac4b53eb0ff6 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:37.442443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2a4c6-2305-415c-a3bc-ac4b53eb0ff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:37.442929 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d2a4c6-2305-415c-a3bc-ac4b53eb0ff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:37.464729 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2a4c6-2305-415c-a3bc-ac4b53eb0ff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:37.464729 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d2a4c6-2305-415c-a3bc-ac4b53eb0ff6 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:37.473103 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1317/2624] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:11:37.570389 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447b8de-16ad-4062-ba95-b18a1baa2b22 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:37.571552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447b8de-16ad-4062-ba95-b18a1baa2b22 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:37.571958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3447b8de-16ad-4062-ba95-b18a1baa2b22 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:37.573380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1308/2625] 10.222.0.67 () {54 vars in 921 bytes} [Mon Sep 26 18:11:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 26 18:11:37.580560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95486942-bd98-4142-82df-31f83f6355cb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:37.580905 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95486942-bd98-4142-82df-31f83f6355cb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:37.581224 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95486942-bd98-4142-82df-31f83f6355cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:37.613931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-95486942-bd98-4142-82df-31f83f6355cb None None] MFA Rules not processed for user `b9ed29a3f6be4d64bb3751f9e95eb56b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:37.615180 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-95486942-bd98-4142-82df-31f83f6355cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:37.626918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1318/2626] 10.222.0.67 () {58 vars in 1018 bytes} [Mon Sep 26 18:11:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:11:37.646413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447b8de-16ad-4062-ba95-b18a1baa2b22 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:37.648484 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3447b8de-16ad-4062-ba95-b18a1baa2b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:37.665589 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3447b8de-16ad-4062-ba95-b18a1baa2b22 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:37.682558 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c10a8b-6f60-44a0-a551-5c7f24e76e7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:37.682558 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c10a8b-6f60-44a0-a551-5c7f24e76e7e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:37.682558 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c10a8b-6f60-44a0-a551-5c7f24e76e7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:37.682558 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c10a8b-6f60-44a0-a551-5c7f24e76e7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:37.688035 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c10a8b-6f60-44a0-a551-5c7f24e76e7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:37.689522 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c10a8b-6f60-44a0-a551-5c7f24e76e7e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:37.702058 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1309/2627] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:11:38.877052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95486942-bd98-4142-82df-31f83f6355cb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:38.879415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-95486942-bd98-4142-82df-31f83f6355cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:38.993910 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95486942-bd98-4142-82df-31f83f6355cb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:39.000667 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d55379-9551-4b92-983a-00a5edce817b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:39.001387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d55379-9551-4b92-983a-00a5edce817b service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:39.001667 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d55379-9551-4b92-983a-00a5edce817b service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:39.001997 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d55379-9551-4b92-983a-00a5edce817b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:39.005214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a3d55379-9551-4b92-983a-00a5edce817b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:39.019600 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d55379-9551-4b92-983a-00a5edce817b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:39.021288 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d55379-9551-4b92-983a-00a5edce817b service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:39.033078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1319/2628] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:11:40.524988 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c10a8b-6f60-44a0-a551-5c7f24e76e7e service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:40.567060 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:40.598045 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c10a8b-6f60-44a0-a551-5c7f24e76e7e service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:40.603755 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e31c3-79f2-4a2c-a710-05cbc7a2d316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:40.604403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e31c3-79f2-4a2c-a710-05cbc7a2d316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:40.604686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e31c3-79f2-4a2c-a710-05cbc7a2d316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:40.604963 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e31c3-79f2-4a2c-a710-05cbc7a2d316 admin admin] PATH_INFO: `/v3/users/614fdacf9d7e46c496beaa774c9c3b42` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:40.606103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e31c3-79f2-4a2c-a710-05cbc7a2d316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=614fdacf9d7e46c496beaa774c9c3b42)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:40.608297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e31c3-79f2-4a2c-a710-05cbc7a2d316 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:40.803421 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-aa5e31c3-79f2-4a2c-a710-05cbc7a2d316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '614fdacf9d7e46c496beaa774c9c3b42'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:40.889300 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-aa5e31c3-79f2-4a2c-a710-05cbc7a2d316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '614fdacf9d7e46c496beaa774c9c3b42'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:40.890141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-aa5e31c3-79f2-4a2c-a710-05cbc7a2d316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '614fdacf9d7e46c496beaa774c9c3b42'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:40.891631 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-aa5e31c3-79f2-4a2c-a710-05cbc7a2d316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '614fdacf9d7e46c496beaa774c9c3b42'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:40.904222 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-aa5e31c3-79f2-4a2c-a710-05cbc7a2d316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '614fdacf9d7e46c496beaa774c9c3b42'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:40.930449 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1310/2629] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:11:40 2022] DELETE /identity/v3/users/614fdacf9d7e46c496beaa774c9c3b42 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:11:40.934924 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b8c4a-cd47-441f-b557-f54b86ddbad5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:40.935311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b8c4a-cd47-441f-b557-f54b86ddbad5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:40.935541 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b8c4a-cd47-441f-b557-f54b86ddbad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:40.956188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-660b8c4a-cd47-441f-b557-f54b86ddbad5 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:40.957462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-660b8c4a-cd47-441f-b557-f54b86ddbad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:41.023504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1320/2630] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:41.035384 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e31c3-79f2-4a2c-a710-05cbc7a2d316 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:41.036167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-aa5e31c3-79f2-4a2c-a710-05cbc7a2d316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:41.100043 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e31c3-79f2-4a2c-a710-05cbc7a2d316 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:41.104080 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c410a-8fe0-4c68-a841-5ca64e730b1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:41.105036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c410a-8fe0-4c68-a841-5ca64e730b1c service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:41.105323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c410a-8fe0-4c68-a841-5ca64e730b1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:41.105771 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1c410a-8fe0-4c68-a841-5ca64e730b1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:41.116241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c410a-8fe0-4c68-a841-5ca64e730b1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:41.121373 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1c410a-8fe0-4c68-a841-5ca64e730b1c service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:41.128585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1311/2631] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:11:41.137990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa678d-6019-48bd-926a-57095ebee5f1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:41.138372 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa678d-6019-48bd-926a-57095ebee5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:41.138657 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa678d-6019-48bd-926a-57095ebee5f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:41.160408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0daa678d-6019-48bd-926a-57095ebee5f1 None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:41.161629 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0daa678d-6019-48bd-926a-57095ebee5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:41.167476 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:41.243427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1321/2632] 10.222.0.67 () {58 vars in 1009 bytes} [Mon Sep 26 18:11:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:41.267489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c410a-8fe0-4c68-a841-5ca64e730b1c service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:41.267489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4c1c410a-8fe0-4c68-a841-5ca64e730b1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:41.319483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:41.336053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1c410a-8fe0-4c68-a841-5ca64e730b1c service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:41.342269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af910a98-9303-4570-92d7-f8583b933d0d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:41.343024 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af910a98-9303-4570-92d7-f8583b933d0d service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:41.343560 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af910a98-9303-4570-92d7-f8583b933d0d service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:41.344639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af910a98-9303-4570-92d7-f8583b933d0d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:41.355580 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af910a98-9303-4570-92d7-f8583b933d0d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:41.362040 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af910a98-9303-4570-92d7-f8583b933d0d service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:41.371321 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1312/2633] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:11:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:11:41.688455 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa678d-6019-48bd-926a-57095ebee5f1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:41.691460 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0daa678d-6019-48bd-926a-57095ebee5f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:41.711008 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa678d-6019-48bd-926a-57095ebee5f1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:41.716315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1436711-c9b4-447f-a07e-d7eeabeb229e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:41.733691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1436711-c9b4-447f-a07e-d7eeabeb229e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:41.734597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1436711-c9b4-447f-a07e-d7eeabeb229e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:41.735109 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1436711-c9b4-447f-a07e-d7eeabeb229e admin admin] PATH_INFO: `/v3/projects/574db4595c454f0fbd8672928b7ca212` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:41.736564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1436711-c9b4-447f-a07e-d7eeabeb229e admin admin] RBAC: Authorizing `identity:delete_project(project_id=574db4595c454f0fbd8672928b7ca212)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:41.738251 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1436711-c9b4-447f-a07e-d7eeabeb229e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:41.898508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f1436711-c9b4-447f-a07e-d7eeabeb229e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '574db4595c454f0fbd8672928b7ca212', 'request_id': 'req-f1436711-c9b4-447f-a07e-d7eeabeb229e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:41.909551 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f1436711-c9b4-447f-a07e-d7eeabeb229e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '574db4595c454f0fbd8672928b7ca212', 'request_id': 'req-f1436711-c9b4-447f-a07e-d7eeabeb229e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:41.981988 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f1436711-c9b4-447f-a07e-d7eeabeb229e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '574db4595c454f0fbd8672928b7ca212', 'request_id': 'req-f1436711-c9b4-447f-a07e-d7eeabeb229e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:41.996830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f1436711-c9b4-447f-a07e-d7eeabeb229e admin admin] The token cache is being invalidate because project 574db4595c454f0fbd8672928b7ca212 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:11:41.997172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f1436711-c9b4-447f-a07e-d7eeabeb229e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:41.997776 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f1436711-c9b4-447f-a07e-d7eeabeb229e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:41.999700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1322/2634] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:11:41 2022] DELETE /identity/v3/projects/574db4595c454f0fbd8672928b7ca212 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:42.003841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af910a98-9303-4570-92d7-f8583b933d0d service swift] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:42.027350 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af910a98-9303-4570-92d7-f8583b933d0d service swift] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:42.120690 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96abd179-f720-4f14-875e-5c3e20381487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:42.121454 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abd179-f720-4f14-875e-5c3e20381487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:42.121737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abd179-f720-4f14-875e-5c3e20381487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:42.122016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abd179-f720-4f14-875e-5c3e20381487 admin admin] PATH_INFO: `/v3/users/abae3fca6d734473bdeb0f763351272d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:42.123415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96abd179-f720-4f14-875e-5c3e20381487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abae3fca6d734473bdeb0f763351272d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:42.126385 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96abd179-f720-4f14-875e-5c3e20381487 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:42.338468 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-96abd179-f720-4f14-875e-5c3e20381487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abae3fca6d734473bdeb0f763351272d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:42.408914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-96abd179-f720-4f14-875e-5c3e20381487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abae3fca6d734473bdeb0f763351272d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:42.409598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-96abd179-f720-4f14-875e-5c3e20381487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abae3fca6d734473bdeb0f763351272d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:42.410224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-96abd179-f720-4f14-875e-5c3e20381487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abae3fca6d734473bdeb0f763351272d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:42.418935 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-96abd179-f720-4f14-875e-5c3e20381487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abae3fca6d734473bdeb0f763351272d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:42.441960 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1313/2635] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:11:42 2022] DELETE /identity/v3/users/abae3fca6d734473bdeb0f763351272d => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:42.953865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1436711-c9b4-447f-a07e-d7eeabeb229e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:42.955446 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f1436711-c9b4-447f-a07e-d7eeabeb229e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:42.990666 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1436711-c9b4-447f-a07e-d7eeabeb229e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:43.089223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa3bca-9d78-40d0-96bf-13d5484ca170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:43.090045 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3bca-9d78-40d0-96bf-13d5484ca170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:43.090558 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3bca-9d78-40d0-96bf-13d5484ca170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:43.091031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa3bca-9d78-40d0-96bf-13d5484ca170 admin admin] PATH_INFO: `/v3/projects/7404835c03f0473d8657a3c979baa3e2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:43.092750 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa3bca-9d78-40d0-96bf-13d5484ca170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7404835c03f0473d8657a3c979baa3e2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:43.095542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fa3bca-9d78-40d0-96bf-13d5484ca170 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:43.205848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:43.221410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-59fa3bca-9d78-40d0-96bf-13d5484ca170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7404835c03f0473d8657a3c979baa3e2', 'request_id': 'req-59fa3bca-9d78-40d0-96bf-13d5484ca170'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.229341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-59fa3bca-9d78-40d0-96bf-13d5484ca170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7404835c03f0473d8657a3c979baa3e2', 'request_id': 'req-59fa3bca-9d78-40d0-96bf-13d5484ca170'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.323502 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-59fa3bca-9d78-40d0-96bf-13d5484ca170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7404835c03f0473d8657a3c979baa3e2', 'request_id': 'req-59fa3bca-9d78-40d0-96bf-13d5484ca170'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.340138 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-59fa3bca-9d78-40d0-96bf-13d5484ca170 admin admin] The token cache is being invalidate because project 7404835c03f0473d8657a3c979baa3e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:11:43.340413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-59fa3bca-9d78-40d0-96bf-13d5484ca170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.341136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-59fa3bca-9d78-40d0-96bf-13d5484ca170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:43.342681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1323/2636] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:11:42 2022] DELETE /identity/v3/projects/7404835c03f0473d8657a3c979baa3e2 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:43.352479 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf514132-2b31-4eb9-b685-15d23e9e6e86 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:43.353052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf514132-2b31-4eb9-b685-15d23e9e6e86 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:43.353150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf514132-2b31-4eb9-b685-15d23e9e6e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:43.379893 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cf514132-2b31-4eb9-b685-15d23e9e6e86 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:43.382036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cf514132-2b31-4eb9-b685-15d23e9e6e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:43.479763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1314/2637] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:11:43.492479 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9bf02-4055-47ef-9e81-540bdb881c7c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:43.492957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9bf02-4055-47ef-9e81-540bdb881c7c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:43.493413 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9bf02-4055-47ef-9e81-540bdb881c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:43.520542 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-20c9bf02-4055-47ef-9e81-540bdb881c7c None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:43.521972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-20c9bf02-4055-47ef-9e81-540bdb881c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:43.532461 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1324/2638] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:43.536918 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf514132-2b31-4eb9-b685-15d23e9e6e86 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:43.546997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:43.550679 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf514132-2b31-4eb9-b685-15d23e9e6e86 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:43.558412 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5807af-ea98-4a15-bf57-236ba7582f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:43.562949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5807af-ea98-4a15-bf57-236ba7582f7a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:43.562949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5807af-ea98-4a15-bf57-236ba7582f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:43.562949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5807af-ea98-4a15-bf57-236ba7582f7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:43.562949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5807af-ea98-4a15-bf57-236ba7582f7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:11:43.562949 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5807af-ea98-4a15-bf57-236ba7582f7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:43.565490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5807af-ea98-4a15-bf57-236ba7582f7a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:43.579315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1315/2639] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:11:43.585396 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9bf02-4055-47ef-9e81-540bdb881c7c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:43.599293 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9bf02-4055-47ef-9e81-540bdb881c7c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:43.607071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a604f-fd27-441f-9dd6-1b455f9b7611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:43.607730 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a604f-fd27-441f-9dd6-1b455f9b7611 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:43.608005 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a604f-fd27-441f-9dd6-1b455f9b7611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:43.608289 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9a604f-fd27-441f-9dd6-1b455f9b7611 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:43.609498 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a604f-fd27-441f-9dd6-1b455f9b7611 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:43.611526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9a604f-fd27-441f-9dd6-1b455f9b7611 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:43.700964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1325/2640] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:11:43 2022] POST /identity/v3/projects => generated 375 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:43.707223 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5807af-ea98-4a15-bf57-236ba7582f7a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:43.719883 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5807af-ea98-4a15-bf57-236ba7582f7a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:43.823858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6026b-ec50-4b82-8301-b6142305abd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:43.824495 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6026b-ec50-4b82-8301-b6142305abd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:43.824786 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6026b-ec50-4b82-8301-b6142305abd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:43.825055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed6026b-ec50-4b82-8301-b6142305abd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:43.825696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6026b-ec50-4b82-8301-b6142305abd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:43.828026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed6026b-ec50-4b82-8301-b6142305abd7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:43.941852 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1316/2641] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:11:43 2022] POST /identity/v3/users => generated 288 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:43.969156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a604f-fd27-441f-9dd6-1b455f9b7611 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:43.972581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9a604f-fd27-441f-9dd6-1b455f9b7611 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:43.985801 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44114722-8d40-4930-8b09-507105187c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:43.986673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44114722-8d40-4930-8b09-507105187c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:43.986963 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44114722-8d40-4930-8b09-507105187c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:43.987239 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44114722-8d40-4930-8b09-507105187c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:43.988004 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44114722-8d40-4930-8b09-507105187c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:43.990177 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44114722-8d40-4930-8b09-507105187c1c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:44.006393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1326/2642] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:11:44.007166 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6026b-ec50-4b82-8301-b6142305abd7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:44.022201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed6026b-ec50-4b82-8301-b6142305abd7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:44.027794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfadfa20-a348-4720-b383-501a67d4bf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:44.028592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfadfa20-a348-4720-b383-501a67d4bf31 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:44.028948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfadfa20-a348-4720-b383-501a67d4bf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:44.029230 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfadfa20-a348-4720-b383-501a67d4bf31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:44.029920 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfadfa20-a348-4720-b383-501a67d4bf31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:44.031987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfadfa20-a348-4720-b383-501a67d4bf31 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:44.049681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44114722-8d40-4930-8b09-507105187c1c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:44.050517 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1317/2643] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:11:44.061663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44114722-8d40-4930-8b09-507105187c1c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:44.069148 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7359a-8735-4f58-8336-5ca46ba62846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:44.069833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7359a-8735-4f58-8336-5ca46ba62846 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:44.070110 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7359a-8735-4f58-8336-5ca46ba62846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:44.070389 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e7359a-8735-4f58-8336-5ca46ba62846 admin admin] PATH_INFO: `/v3/projects/e2faa310cf284a5eb1ee6905e03c0c14/users/5749415f50f0492b97731f01393e8e1c/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:44.087416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7359a-8735-4f58-8336-5ca46ba62846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2faa310cf284a5eb1ee6905e03c0c14, user_id=5749415f50f0492b97731f01393e8e1c, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:44.090293 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e7359a-8735-4f58-8336-5ca46ba62846 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:44.174392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1327/2644] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:11:44 2022] PUT /identity/v3/projects/e2faa310cf284a5eb1ee6905e03c0c14/users/5749415f50f0492b97731f01393e8e1c/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:11:44.194649 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd21359-f82e-41ce-9615-e0fc3e96d797 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:44.194960 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd21359-f82e-41ce-9615-e0fc3e96d797 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:44.195231 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd21359-f82e-41ce-9615-e0fc3e96d797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:44.257171 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3cd21359-f82e-41ce-9615-e0fc3e96d797 None None] MFA Rules not processed for user `5749415f50f0492b97731f01393e8e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:44.258346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3cd21359-f82e-41ce-9615-e0fc3e96d797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:44.371046 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1318/2645] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:44.380669 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7359a-8735-4f58-8336-5ca46ba62846 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:44.382387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-31e7359a-8735-4f58-8336-5ca46ba62846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:44.520893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e7359a-8735-4f58-8336-5ca46ba62846 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:44.529853 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a414f79e-ef4a-4acd-97ba-35b406301669 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:44.530492 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414f79e-ef4a-4acd-97ba-35b406301669 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:44.531392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414f79e-ef4a-4acd-97ba-35b406301669 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:44.535110 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a414f79e-ef4a-4acd-97ba-35b406301669 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:44.545190 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414f79e-ef4a-4acd-97ba-35b406301669 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:44.546684 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a414f79e-ef4a-4acd-97ba-35b406301669 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:44.556695 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1328/2646] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:11:49.344208 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd21359-f82e-41ce-9615-e0fc3e96d797 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:49.437976 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd21359-f82e-41ce-9615-e0fc3e96d797 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:49.445950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d74eee-af37-4739-b052-94b94e7c4c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:49.447213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d74eee-af37-4739-b052-94b94e7c4c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:49.447873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d74eee-af37-4739-b052-94b94e7c4c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:49.448386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d74eee-af37-4739-b052-94b94e7c4c1a admin admin] PATH_INFO: `/v3/users/5749415f50f0492b97731f01393e8e1c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:49.450189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d74eee-af37-4739-b052-94b94e7c4c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5749415f50f0492b97731f01393e8e1c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:49.453996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d74eee-af37-4739-b052-94b94e7c4c1a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:49.456905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:49.731708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-72d74eee-af37-4739-b052-94b94e7c4c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5749415f50f0492b97731f01393e8e1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:49.814910 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-72d74eee-af37-4739-b052-94b94e7c4c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5749415f50f0492b97731f01393e8e1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:49.815705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-72d74eee-af37-4739-b052-94b94e7c4c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5749415f50f0492b97731f01393e8e1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:49.816719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-72d74eee-af37-4739-b052-94b94e7c4c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5749415f50f0492b97731f01393e8e1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:49.830483 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-72d74eee-af37-4739-b052-94b94e7c4c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5749415f50f0492b97731f01393e8e1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:49.868444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1319/2647] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:11:49 2022] DELETE /identity/v3/users/5749415f50f0492b97731f01393e8e1c => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:49.877444 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226dac2-89c7-44b0-a027-5617a00ca6eb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:49.877444 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226dac2-89c7-44b0-a027-5617a00ca6eb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:49.877444 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226dac2-89c7-44b0-a027-5617a00ca6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:49.895480 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8226dac2-89c7-44b0-a027-5617a00ca6eb None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:49.896960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8226dac2-89c7-44b0-a027-5617a00ca6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:49.993618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:50.003878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1329/2648] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:11:50.020101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d74eee-af37-4739-b052-94b94e7c4c1a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:50.022788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-72d74eee-af37-4739-b052-94b94e7c4c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:50.116964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d74eee-af37-4739-b052-94b94e7c4c1a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:50.128654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdf0b32-a7cd-475a-8775-1b80c961ec92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:50.128654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf0b32-a7cd-475a-8775-1b80c961ec92 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:50.128654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf0b32-a7cd-475a-8775-1b80c961ec92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:50.128654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdf0b32-a7cd-475a-8775-1b80c961ec92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:50.136478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf0b32-a7cd-475a-8775-1b80c961ec92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:50.139239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdf0b32-a7cd-475a-8775-1b80c961ec92 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:50.157315 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1320/2649] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:11:51.938198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226dac2-89c7-44b0-a027-5617a00ca6eb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:51.943469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8226dac2-89c7-44b0-a027-5617a00ca6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:51.954279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226dac2-89c7-44b0-a027-5617a00ca6eb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:51.960126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb16b6-f154-4a7f-888a-7d1a545a0bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:51.961016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb16b6-f154-4a7f-888a-7d1a545a0bd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:51.961515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb16b6-f154-4a7f-888a-7d1a545a0bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:51.961980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb16b6-f154-4a7f-888a-7d1a545a0bd3 admin admin] PATH_INFO: `/v3/projects/e2faa310cf284a5eb1ee6905e03c0c14` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:51.964077 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb16b6-f154-4a7f-888a-7d1a545a0bd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2faa310cf284a5eb1ee6905e03c0c14)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:51.968113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb16b6-f154-4a7f-888a-7d1a545a0bd3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:52.209089 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-74fb16b6-f154-4a7f-888a-7d1a545a0bd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2faa310cf284a5eb1ee6905e03c0c14', 'request_id': 'req-74fb16b6-f154-4a7f-888a-7d1a545a0bd3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:52.216473 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-74fb16b6-f154-4a7f-888a-7d1a545a0bd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2faa310cf284a5eb1ee6905e03c0c14', 'request_id': 'req-74fb16b6-f154-4a7f-888a-7d1a545a0bd3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:52.381311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-74fb16b6-f154-4a7f-888a-7d1a545a0bd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2faa310cf284a5eb1ee6905e03c0c14', 'request_id': 'req-74fb16b6-f154-4a7f-888a-7d1a545a0bd3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:52.396571 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-74fb16b6-f154-4a7f-888a-7d1a545a0bd3 admin admin] The token cache is being invalidate because project e2faa310cf284a5eb1ee6905e03c0c14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:11:52.396871 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-74fb16b6-f154-4a7f-888a-7d1a545a0bd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:52.397578 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-74fb16b6-f154-4a7f-888a-7d1a545a0bd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:52.399186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1330/2650] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:11:51 2022] DELETE /identity/v3/projects/e2faa310cf284a5eb1ee6905e03c0c14 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:52.404338 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f53e0-a37c-41a2-88d5-b15882f9619b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:52.404975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f53e0-a37c-41a2-88d5-b15882f9619b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:52.404975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f53e0-a37c-41a2-88d5-b15882f9619b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:52.426966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ff3f53e0-a37c-41a2-88d5-b15882f9619b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:52.428228 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ff3f53e0-a37c-41a2-88d5-b15882f9619b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:52.521234 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1321/2651] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:52.530225 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f4e90-23e2-4d0b-919f-eb33fab19cc1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:52.533256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f4e90-23e2-4d0b-919f-eb33fab19cc1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:52.533256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9f4e90-23e2-4d0b-919f-eb33fab19cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:52.561143 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3a9f4e90-23e2-4d0b-919f-eb33fab19cc1 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:52.563838 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3a9f4e90-23e2-4d0b-919f-eb33fab19cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:52.576456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1331/2652] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:52.581331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f53e0-a37c-41a2-88d5-b15882f9619b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:52.591267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f53e0-a37c-41a2-88d5-b15882f9619b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:52.596702 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397db433-b711-4b04-8048-a13ff6a26a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:52.597344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397db433-b711-4b04-8048-a13ff6a26a16 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:52.597659 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397db433-b711-4b04-8048-a13ff6a26a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:52.597929 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397db433-b711-4b04-8048-a13ff6a26a16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:52.598881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397db433-b711-4b04-8048-a13ff6a26a16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:11:52.599410 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397db433-b711-4b04-8048-a13ff6a26a16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:52.600563 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397db433-b711-4b04-8048-a13ff6a26a16 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:52.619736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1322/2653] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:11:52.625060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f4e90-23e2-4d0b-919f-eb33fab19cc1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:52.634560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9f4e90-23e2-4d0b-919f-eb33fab19cc1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:52.639389 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa7466-0752-485e-bc61-e87e2c81cb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:52.670887 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa7466-0752-485e-bc61-e87e2c81cb19 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:52.670887 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa7466-0752-485e-bc61-e87e2c81cb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:52.670887 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa7466-0752-485e-bc61-e87e2c81cb19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:52.670887 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa7466-0752-485e-bc61-e87e2c81cb19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:52.670887 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa7466-0752-485e-bc61-e87e2c81cb19 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:52.825267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1332/2654] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:11:52 2022] POST /identity/v3/projects => generated 383 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:52.826584 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397db433-b711-4b04-8048-a13ff6a26a16 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:52.844137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397db433-b711-4b04-8048-a13ff6a26a16 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:52.850180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:52.858364 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa7466-0752-485e-bc61-e87e2c81cb19 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:52.861387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b6fa7466-0752-485e-bc61-e87e2c81cb19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:52.937069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdc0ad-bcb6-4eb7-9e04-acb6e6b7d2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:52.938463 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdc0ad-bcb6-4eb7-9e04-acb6e6b7d2ca admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:52.938902 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdc0ad-bcb6-4eb7-9e04-acb6e6b7d2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:52.939345 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cdc0ad-bcb6-4eb7-9e04-acb6e6b7d2ca admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:52.940352 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdc0ad-bcb6-4eb7-9e04-acb6e6b7d2ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:52.949637 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cdc0ad-bcb6-4eb7-9e04-acb6e6b7d2ca admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:52.959437 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa7466-0752-485e-bc61-e87e2c81cb19 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:52.966175 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43bc3f-db13-496a-8691-26b6ae3517b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:52.967409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43bc3f-db13-496a-8691-26b6ae3517b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:52.968759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43bc3f-db13-496a-8691-26b6ae3517b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:52.969517 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43bc3f-db13-496a-8691-26b6ae3517b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:52.973870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ee43bc3f-db13-496a-8691-26b6ae3517b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:53.059063 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43bc3f-db13-496a-8691-26b6ae3517b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:53.060598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee43bc3f-db13-496a-8691-26b6ae3517b1 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:53.076218 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1333/2655] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:11:53.142059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1323/2656] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:11:52 2022] POST /identity/v3/users => generated 292 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:53.146692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43bc3f-db13-496a-8691-26b6ae3517b1 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:53.158610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43bc3f-db13-496a-8691-26b6ae3517b1 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:53.163627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac115d-c7de-427d-b447-569af41d3daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:53.164590 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac115d-c7de-427d-b447-569af41d3daf admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:53.164887 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac115d-c7de-427d-b447-569af41d3daf admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:53.165291 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ac115d-c7de-427d-b447-569af41d3daf admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:53.165804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac115d-c7de-427d-b447-569af41d3daf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:53.167460 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ac115d-c7de-427d-b447-569af41d3daf admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:53.169158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:53.180385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1334/2657] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:11:53.185532 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdc0ad-bcb6-4eb7-9e04-acb6e6b7d2ca admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:53.206829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cdc0ad-bcb6-4eb7-9e04-acb6e6b7d2ca admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:53.212686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbaaae7-67f7-45fc-ac47-9b6a79dec0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:53.213447 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaaae7-67f7-45fc-ac47-9b6a79dec0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:53.213648 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaaae7-67f7-45fc-ac47-9b6a79dec0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:53.213891 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbaaae7-67f7-45fc-ac47-9b6a79dec0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:53.214561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaaae7-67f7-45fc-ac47-9b6a79dec0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:53.216252 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbaaae7-67f7-45fc-ac47-9b6a79dec0e7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:53.229748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1324/2658] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:11:53.237540 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac115d-c7de-427d-b447-569af41d3daf admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:53.255225 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ac115d-c7de-427d-b447-569af41d3daf admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:53.256330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb6834-5881-4480-99f6-ff360a9004f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:53.257057 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb6834-5881-4480-99f6-ff360a9004f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:53.257346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb6834-5881-4480-99f6-ff360a9004f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:53.257854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb6834-5881-4480-99f6-ff360a9004f9 admin admin] PATH_INFO: `/v3/projects/db64061b1b004807a888bcc7d88e62ce/users/e1dbe2129f964b4e8abd7c67e233eaeb/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:53.283949 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb6834-5881-4480-99f6-ff360a9004f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db64061b1b004807a888bcc7d88e62ce, user_id=e1dbe2129f964b4e8abd7c67e233eaeb, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:53.287755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb6834-5881-4480-99f6-ff360a9004f9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:53.401685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1335/2659] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:11:53 2022] PUT /identity/v3/projects/db64061b1b004807a888bcc7d88e62ce/users/e1dbe2129f964b4e8abd7c67e233eaeb/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:11:53.414919 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb125f4-74d8-4106-9d93-141a988e7ece None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:53.415627 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb125f4-74d8-4106-9d93-141a988e7ece None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:53.415627 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb125f4-74d8-4106-9d93-141a988e7ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:53.451279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7fb125f4-74d8-4106-9d93-141a988e7ece None None] MFA Rules not processed for user `e1dbe2129f964b4e8abd7c67e233eaeb`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:53.453688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7fb125f4-74d8-4106-9d93-141a988e7ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:53.558682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1325/2660] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:11:53.597512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb6834-5881-4480-99f6-ff360a9004f9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:53.600615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0fdb6834-5881-4480-99f6-ff360a9004f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:53.701653 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb6834-5881-4480-99f6-ff360a9004f9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:53.707918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294f2b81-01cc-40ff-b7ff-dc389747e9c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:53.708865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f2b81-01cc-40ff-b7ff-dc389747e9c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:53.709313 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f2b81-01cc-40ff-b7ff-dc389747e9c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:53.709730 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294f2b81-01cc-40ff-b7ff-dc389747e9c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:53.723351 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294f2b81-01cc-40ff-b7ff-dc389747e9c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:53.726906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294f2b81-01cc-40ff-b7ff-dc389747e9c3 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:53.738034 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1336/2661] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:11:57.099523 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb125f4-74d8-4106-9d93-141a988e7ece None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:57.182928 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb125f4-74d8-4106-9d93-141a988e7ece None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:57.188454 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f40e8-3acf-465d-8a05-ccb0917f4ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:57.191182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f40e8-3acf-465d-8a05-ccb0917f4ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:57.191182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f40e8-3acf-465d-8a05-ccb0917f4ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:57.191182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76f40e8-3acf-465d-8a05-ccb0917f4ff7 admin admin] PATH_INFO: `/v3/users/e1dbe2129f964b4e8abd7c67e233eaeb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:57.191182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f40e8-3acf-465d-8a05-ccb0917f4ff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1dbe2129f964b4e8abd7c67e233eaeb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:57.192741 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76f40e8-3acf-465d-8a05-ccb0917f4ff7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:57.222485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:57.406492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b76f40e8-3acf-465d-8a05-ccb0917f4ff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1dbe2129f964b4e8abd7c67e233eaeb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:57.461374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b76f40e8-3acf-465d-8a05-ccb0917f4ff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1dbe2129f964b4e8abd7c67e233eaeb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:57.462482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b76f40e8-3acf-465d-8a05-ccb0917f4ff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1dbe2129f964b4e8abd7c67e233eaeb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:57.463575 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b76f40e8-3acf-465d-8a05-ccb0917f4ff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1dbe2129f964b4e8abd7c67e233eaeb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:57.477488 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b76f40e8-3acf-465d-8a05-ccb0917f4ff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1dbe2129f964b4e8abd7c67e233eaeb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:57.500701 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1326/2662] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:11:57 2022] DELETE /identity/v3/users/e1dbe2129f964b4e8abd7c67e233eaeb => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:11:57.505658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9844df3-62e7-425a-bfc6-8fc07ac0834b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:57.506227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9844df3-62e7-425a-bfc6-8fc07ac0834b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:57.506589 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9844df3-62e7-425a-bfc6-8fc07ac0834b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:57.534284 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d9844df3-62e7-425a-bfc6-8fc07ac0834b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:57.535896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d9844df3-62e7-425a-bfc6-8fc07ac0834b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:57.627386 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1337/2663] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:57.634992 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f40e8-3acf-465d-8a05-ccb0917f4ff7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:57.636857 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b76f40e8-3acf-465d-8a05-ccb0917f4ff7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:57.729774 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76f40e8-3acf-465d-8a05-ccb0917f4ff7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:57.735969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05461bda-fbcd-49cb-926c-97274f04dfef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:57.736783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05461bda-fbcd-49cb-926c-97274f04dfef service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:57.737278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05461bda-fbcd-49cb-926c-97274f04dfef service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:57.738434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05461bda-fbcd-49cb-926c-97274f04dfef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:57.751890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05461bda-fbcd-49cb-926c-97274f04dfef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:57.753219 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05461bda-fbcd-49cb-926c-97274f04dfef service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:57.768947 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1327/2664] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:11:58.321193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9844df3-62e7-425a-bfc6-8fc07ac0834b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:58.323311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d9844df3-62e7-425a-bfc6-8fc07ac0834b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:58.342643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9844df3-62e7-425a-bfc6-8fc07ac0834b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:58.348494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d6136-636d-4497-a00c-73b6aad7e5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:58.349809 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d6136-636d-4497-a00c-73b6aad7e5a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:58.350611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d6136-636d-4497-a00c-73b6aad7e5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:58.351215 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d6136-636d-4497-a00c-73b6aad7e5a0 admin admin] PATH_INFO: `/v3/projects/db64061b1b004807a888bcc7d88e62ce` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:58.354319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d6136-636d-4497-a00c-73b6aad7e5a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db64061b1b004807a888bcc7d88e62ce)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:58.358079 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d6136-636d-4497-a00c-73b6aad7e5a0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:58.516933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bc8d6136-636d-4497-a00c-73b6aad7e5a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db64061b1b004807a888bcc7d88e62ce', 'request_id': 'req-bc8d6136-636d-4497-a00c-73b6aad7e5a0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:58.528048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bc8d6136-636d-4497-a00c-73b6aad7e5a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db64061b1b004807a888bcc7d88e62ce', 'request_id': 'req-bc8d6136-636d-4497-a00c-73b6aad7e5a0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:58.614950 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bc8d6136-636d-4497-a00c-73b6aad7e5a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db64061b1b004807a888bcc7d88e62ce', 'request_id': 'req-bc8d6136-636d-4497-a00c-73b6aad7e5a0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:58.625320 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:58.634464 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bc8d6136-636d-4497-a00c-73b6aad7e5a0 admin admin] The token cache is being invalidate because project db64061b1b004807a888bcc7d88e62ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:11:58.635261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bc8d6136-636d-4497-a00c-73b6aad7e5a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:58.637125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bc8d6136-636d-4497-a00c-73b6aad7e5a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:11:58.638931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1338/2665] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:11:58 2022] DELETE /identity/v3/projects/db64061b1b004807a888bcc7d88e62ce => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:11:58.646426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71329327-5db4-4e0e-b862-99cff5a7b402 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:58.647192 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71329327-5db4-4e0e-b862-99cff5a7b402 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:58.648287 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71329327-5db4-4e0e-b862-99cff5a7b402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:58.678120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-71329327-5db4-4e0e-b862-99cff5a7b402 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:58.679733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-71329327-5db4-4e0e-b862-99cff5a7b402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:58.747370 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1328/2666] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:58.814756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4debf53-68ae-4af0-be70-00a1ca88978d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:58.815353 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4debf53-68ae-4af0-be70-00a1ca88978d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:58.815884 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4debf53-68ae-4af0-be70-00a1ca88978d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:58.847868 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f4debf53-68ae-4af0-be70-00a1ca88978d None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:58.849953 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f4debf53-68ae-4af0-be70-00a1ca88978d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:58.861474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1339/2667] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:11:58.867463 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71329327-5db4-4e0e-b862-99cff5a7b402 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:58.886107 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71329327-5db4-4e0e-b862-99cff5a7b402 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:58.893952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33443b1-987d-46ea-b4a2-5fb625297584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:58.894579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33443b1-987d-46ea-b4a2-5fb625297584 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:58.895520 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33443b1-987d-46ea-b4a2-5fb625297584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:58.895854 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33443b1-987d-46ea-b4a2-5fb625297584 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:58.896518 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33443b1-987d-46ea-b4a2-5fb625297584 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:11:58.896820 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33443b1-987d-46ea-b4a2-5fb625297584 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:58.898480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33443b1-987d-46ea-b4a2-5fb625297584 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:58.911960 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1329/2668] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:11:58.916316 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4debf53-68ae-4af0-be70-00a1ca88978d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:58.926307 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4debf53-68ae-4af0-be70-00a1ca88978d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:58.935843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed336c14-8587-4b42-b751-f43dcf791af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:58.936563 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed336c14-8587-4b42-b751-f43dcf791af0 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:58.937835 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed336c14-8587-4b42-b751-f43dcf791af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:58.938211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed336c14-8587-4b42-b751-f43dcf791af0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:58.939008 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed336c14-8587-4b42-b751-f43dcf791af0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:58.941117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed336c14-8587-4b42-b751-f43dcf791af0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:59.046335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1340/2669] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:11:58 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:59.050733 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33443b1-987d-46ea-b4a2-5fb625297584 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:59.061465 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33443b1-987d-46ea-b4a2-5fb625297584 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:59.103383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:59.154358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab01ab-9c35-4dad-8775-f931f01bfd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:59.154972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab01ab-9c35-4dad-8775-f931f01bfd23 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:59.155364 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab01ab-9c35-4dad-8775-f931f01bfd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:59.155626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ab01ab-9c35-4dad-8775-f931f01bfd23 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:59.156274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab01ab-9c35-4dad-8775-f931f01bfd23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:59.158612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ab01ab-9c35-4dad-8775-f931f01bfd23 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:59.198432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b54a7-e444-4429-95c3-4f5df3953c02 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:59.198790 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b54a7-e444-4429-95c3-4f5df3953c02 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:59.199132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b54a7-e444-4429-95c3-4f5df3953c02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:59.221382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-df6b54a7-e444-4429-95c3-4f5df3953c02 None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:59.222590 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-df6b54a7-e444-4429-95c3-4f5df3953c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:59.271720 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1330/2670] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:11:59 2022] POST /identity/v3/users => generated 291 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:11:59.276205 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab01ab-9c35-4dad-8775-f931f01bfd23 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:59.292072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ab01ab-9c35-4dad-8775-f931f01bfd23 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:59.297725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dbd86-8153-4ca4-b9c2-1024e7255cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:59.298513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dbd86-8153-4ca4-b9c2-1024e7255cee admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:59.298893 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dbd86-8153-4ca4-b9c2-1024e7255cee admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:59.299236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1dbd86-8153-4ca4-b9c2-1024e7255cee admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:59.300005 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dbd86-8153-4ca4-b9c2-1024e7255cee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:59.301752 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1dbd86-8153-4ca4-b9c2-1024e7255cee admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:59.315110 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1331/2671] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:11:59.320092 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1341/2672] 10.222.0.67 () {58 vars in 967 bytes} [Mon Sep 26 18:11:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:11:59.324619 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dbd86-8153-4ca4-b9c2-1024e7255cee admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:59.337145 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b54a7-e444-4429-95c3-4f5df3953c02 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:59.339505 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-df6b54a7-e444-4429-95c3-4f5df3953c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:59.344727 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1dbd86-8153-4ca4-b9c2-1024e7255cee admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:59.349192 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abc332-3e8e-4768-87de-6e8747e8c604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:59.350435 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abc332-3e8e-4768-87de-6e8747e8c604 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:59.350731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abc332-3e8e-4768-87de-6e8747e8c604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:59.350999 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abc332-3e8e-4768-87de-6e8747e8c604 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:59.351631 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abc332-3e8e-4768-87de-6e8747e8c604 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:59.353478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78abc332-3e8e-4768-87de-6e8747e8c604 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:59.373538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1332/2673] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:11:59.373538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abc332-3e8e-4768-87de-6e8747e8c604 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:59.389393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78abc332-3e8e-4768-87de-6e8747e8c604 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:59.395736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f280c21-478b-4fa4-864b-d61b9425c112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:59.396519 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f280c21-478b-4fa4-864b-d61b9425c112 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:59.397128 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f280c21-478b-4fa4-864b-d61b9425c112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:59.397438 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f280c21-478b-4fa4-864b-d61b9425c112 admin admin] PATH_INFO: `/v3/projects/518090209d9a462fbf40d6cb86c3c695/users/ecf12b3f54e349d5a8f6587410c79501/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:59.415972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f280c21-478b-4fa4-864b-d61b9425c112 admin admin] RBAC: Authorizing `identity:create_grant(project_id=518090209d9a462fbf40d6cb86c3c695, user_id=ecf12b3f54e349d5a8f6587410c79501, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:59.418923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f280c21-478b-4fa4-864b-d61b9425c112 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:59.479558 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b54a7-e444-4429-95c3-4f5df3953c02 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:59.485457 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d39a4-64e7-4707-8945-bf9cbf287332 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:59.486098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d39a4-64e7-4707-8945-bf9cbf287332 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:59.486374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d39a4-64e7-4707-8945-bf9cbf287332 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:59.486649 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d39a4-64e7-4707-8945-bf9cbf287332 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:59.493846 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1333/2674] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:11:59 2022] PUT /identity/v3/projects/518090209d9a462fbf40d6cb86c3c695/users/ecf12b3f54e349d5a8f6587410c79501/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:11:59.496846 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d39a4-64e7-4707-8945-bf9cbf287332 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:59.500374 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d39a4-64e7-4707-8945-bf9cbf287332 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:59.501654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92895842-a9cc-4efe-8c33-b61c376bbd4a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:59.502023 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92895842-a9cc-4efe-8c33-b61c376bbd4a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:59.502418 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92895842-a9cc-4efe-8c33-b61c376bbd4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:59.517253 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1342/2675] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:11:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:11:59.533336 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-92895842-a9cc-4efe-8c33-b61c376bbd4a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:11:59.535226 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-92895842-a9cc-4efe-8c33-b61c376bbd4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:59.616787 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1334/2676] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:11:59.629623 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d39a4-64e7-4707-8945-bf9cbf287332 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:11:59.631178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-114d39a4-64e7-4707-8945-bf9cbf287332 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:11:59.720031 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d39a4-64e7-4707-8945-bf9cbf287332 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:11:59.742471 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:11:59.787358 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bff4f-c3d5-4c8c-a323-74b1e4736bdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:11:59.788156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bff4f-c3d5-4c8c-a323-74b1e4736bdd service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:11:59.788335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bff4f-c3d5-4c8c-a323-74b1e4736bdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:11:59.789282 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31bff4f-c3d5-4c8c-a323-74b1e4736bdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:11:59.799285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bff4f-c3d5-4c8c-a323-74b1e4736bdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:11:59.806198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31bff4f-c3d5-4c8c-a323-74b1e4736bdd service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:11:59.816531 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1343/2677] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:12:01.318484 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8baea-7020-4448-9f01-a0b25ab5fd6a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:01.319341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8baea-7020-4448-9f01-a0b25ab5fd6a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:01.319824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c8baea-7020-4448-9f01-a0b25ab5fd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:01.357415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f7c8baea-7020-4448-9f01-a0b25ab5fd6a None None] MFA Rules not processed for user `ecf12b3f54e349d5a8f6587410c79501`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:12:01.359136 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f7c8baea-7020-4448-9f01-a0b25ab5fd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:01.440530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:12:01.445717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1335/2678] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:12:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:12:01.463841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bff4f-c3d5-4c8c-a323-74b1e4736bdd service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:01.466352 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b31bff4f-c3d5-4c8c-a323-74b1e4736bdd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:01.598851 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31bff4f-c3d5-4c8c-a323-74b1e4736bdd service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:01.607443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a6bbd-d1fa-4b68-8e19-3914fcce49fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:01.607443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a6bbd-d1fa-4b68-8e19-3914fcce49fc service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:01.607443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a6bbd-d1fa-4b68-8e19-3914fcce49fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:01.607443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a6bbd-d1fa-4b68-8e19-3914fcce49fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:01.618307 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a6bbd-d1fa-4b68-8e19-3914fcce49fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:01.621199 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a6bbd-d1fa-4b68-8e19-3914fcce49fc service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:01.633535 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1344/2679] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:12:01 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:12:01.655297 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8baea-7020-4448-9f01-a0b25ab5fd6a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:01.672677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c8baea-7020-4448-9f01-a0b25ab5fd6a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:01.678766 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41069e19-d122-477a-af89-4c81718acf32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:01.679466 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41069e19-d122-477a-af89-4c81718acf32 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:01.680019 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41069e19-d122-477a-af89-4c81718acf32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:01.680307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41069e19-d122-477a-af89-4c81718acf32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:01.682589 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-41069e19-d122-477a-af89-4c81718acf32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:01.696474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41069e19-d122-477a-af89-4c81718acf32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:01.699863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41069e19-d122-477a-af89-4c81718acf32 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:01.716134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1336/2680] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:12:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:12:03.550000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46578e-7351-4432-ab12-09419aabc8bd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:03.550000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46578e-7351-4432-ab12-09419aabc8bd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:03.550000 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a46578e-7351-4432-ab12-09419aabc8bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:03.571449 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9a46578e-7351-4432-ab12-09419aabc8bd None None] MFA Rules not processed for user `ecf12b3f54e349d5a8f6587410c79501`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:12:03.572759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9a46578e-7351-4432-ab12-09419aabc8bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:03.583223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1345/2681] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:12:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:12:11.499445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41069e19-d122-477a-af89-4c81718acf32 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:11.501514 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-41069e19-d122-477a-af89-4c81718acf32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:11.515507 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41069e19-d122-477a-af89-4c81718acf32 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:11.522036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4fc14-50f3-4749-b1ca-c55e30c1921c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:11.522724 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4fc14-50f3-4749-b1ca-c55e30c1921c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:11.523043 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4fc14-50f3-4749-b1ca-c55e30c1921c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:11.523462 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4fc14-50f3-4749-b1ca-c55e30c1921c admin admin] PATH_INFO: `/v3/users/183fe84502664f49a701a5e9eb4f6cd7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:11.524678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4fc14-50f3-4749-b1ca-c55e30c1921c admin admin] RBAC: Authorizing `identity:delete_user(user_id=183fe84502664f49a701a5e9eb4f6cd7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:11.526596 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4fc14-50f3-4749-b1ca-c55e30c1921c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:11.708521 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cdb4fc14-50f3-4749-b1ca-c55e30c1921c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '183fe84502664f49a701a5e9eb4f6cd7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:11.763429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cdb4fc14-50f3-4749-b1ca-c55e30c1921c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '183fe84502664f49a701a5e9eb4f6cd7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:11.763979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cdb4fc14-50f3-4749-b1ca-c55e30c1921c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '183fe84502664f49a701a5e9eb4f6cd7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:11.765187 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cdb4fc14-50f3-4749-b1ca-c55e30c1921c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '183fe84502664f49a701a5e9eb4f6cd7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:11.773750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cdb4fc14-50f3-4749-b1ca-c55e30c1921c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '183fe84502664f49a701a5e9eb4f6cd7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:11.794853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1337/2682] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:12:11 2022] DELETE /identity/v3/users/183fe84502664f49a701a5e9eb4f6cd7 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:12:12.129802 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46578e-7351-4432-ab12-09419aabc8bd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:12.150029 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a46578e-7351-4432-ab12-09419aabc8bd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:12.228282 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:12.228929 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:12.229205 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:12.229841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e admin admin] PATH_INFO: `/v3/projects/08e4bf78b8f043a598a4b6a1d8acc219` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:12.230971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e admin admin] RBAC: Authorizing `identity:delete_project(project_id=08e4bf78b8f043a598a4b6a1d8acc219)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:12.232931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:12.370376 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08e4bf78b8f043a598a4b6a1d8acc219', 'request_id': 'req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:12.377090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08e4bf78b8f043a598a4b6a1d8acc219', 'request_id': 'req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:12.454398 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08e4bf78b8f043a598a4b6a1d8acc219', 'request_id': 'req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:12.456168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:12:12.469348 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e admin admin] The token cache is being invalidate because project 08e4bf78b8f043a598a4b6a1d8acc219 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:12:12.469636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:12.470306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:12.472692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1346/2683] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:12:12 2022] DELETE /identity/v3/projects/08e4bf78b8f043a598a4b6a1d8acc219 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:12:12.476082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4fc14-50f3-4749-b1ca-c55e30c1921c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:12.477543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cdb4fc14-50f3-4749-b1ca-c55e30c1921c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:12.492101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4fc14-50f3-4749-b1ca-c55e30c1921c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:12.561593 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fe77b-817d-441f-9684-3150f53b3f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:12.562546 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fe77b-817d-441f-9684-3150f53b3f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:12.562704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fe77b-817d-441f-9684-3150f53b3f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:12.562968 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98fe77b-817d-441f-9684-3150f53b3f07 admin admin] PATH_INFO: `/v3/users/62da2adabc3c4ab6959b1b83ea903e38` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:12.564191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fe77b-817d-441f-9684-3150f53b3f07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62da2adabc3c4ab6959b1b83ea903e38)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:12.566121 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98fe77b-817d-441f-9684-3150f53b3f07 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:12.745598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f98fe77b-817d-441f-9684-3150f53b3f07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62da2adabc3c4ab6959b1b83ea903e38'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:12.814445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f98fe77b-817d-441f-9684-3150f53b3f07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62da2adabc3c4ab6959b1b83ea903e38'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:12.815047 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f98fe77b-817d-441f-9684-3150f53b3f07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62da2adabc3c4ab6959b1b83ea903e38'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:12.815693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f98fe77b-817d-441f-9684-3150f53b3f07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62da2adabc3c4ab6959b1b83ea903e38'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:12.830189 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f98fe77b-817d-441f-9684-3150f53b3f07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62da2adabc3c4ab6959b1b83ea903e38'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:12.832307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:12:12.852801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1338/2684] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:12:12 2022] DELETE /identity/v3/users/62da2adabc3c4ab6959b1b83ea903e38 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:12:13.154633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:13.157397 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:13.173849 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55e5d6-c2c1-4f6e-89d5-3a3a6ece856e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:13.249914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:13.250328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:13.250885 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:13.251200 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1 admin admin] PATH_INFO: `/v3/projects/35275742c4a04744be7a7ef07a7f9356` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:13.252400 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35275742c4a04744be7a7ef07a7f9356)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:13.254294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:13.373095 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35275742c4a04744be7a7ef07a7f9356', 'request_id': 'req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:13.389351 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35275742c4a04744be7a7ef07a7f9356', 'request_id': 'req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:13.484729 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35275742c4a04744be7a7ef07a7f9356', 'request_id': 'req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:13.506516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1 admin admin] The token cache is being invalidate because project 35275742c4a04744be7a7ef07a7f9356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:12:13.506918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:13.507523 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:13.509407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1347/2685] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:12:13 2022] DELETE /identity/v3/projects/35275742c4a04744be7a7ef07a7f9356 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:12:22.929055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:12:23.497286 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=9,keystone:SELECT=54,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:12:52.198322 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fe77b-817d-441f-9684-3150f53b3f07 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:52.199826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f98fe77b-817d-441f-9684-3150f53b3f07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:52.258540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98fe77b-817d-441f-9684-3150f53b3f07 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:52.314198 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a49a4-f13b-422f-9b11-09f3a18a5b88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:52.314198 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a49a4-f13b-422f-9b11-09f3a18a5b88 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:52.314198 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a49a4-f13b-422f-9b11-09f3a18a5b88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:52.314198 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404a49a4-f13b-422f-9b11-09f3a18a5b88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:52.323999 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-404a49a4-f13b-422f-9b11-09f3a18a5b88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:52.386145 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a49a4-f13b-422f-9b11-09f3a18a5b88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:52.387580 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404a49a4-f13b-422f-9b11-09f3a18a5b88 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:52.398051 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1339/2686] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:12:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:12:56.244012 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:56.245446 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:56.260521 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d36a61-f54f-4e58-9a22-cc44e0cd58a1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:56.265346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01570afa-0030-4a73-ae58-17852df031ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:56.266037 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01570afa-0030-4a73-ae58-17852df031ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:56.266411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01570afa-0030-4a73-ae58-17852df031ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:56.266653 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01570afa-0030-4a73-ae58-17852df031ee admin admin] PATH_INFO: `/v3/users/612875c8fc7240e5b18fe38a034a3ef5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:56.267921 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01570afa-0030-4a73-ae58-17852df031ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=612875c8fc7240e5b18fe38a034a3ef5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:56.270408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01570afa-0030-4a73-ae58-17852df031ee admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:56.457214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-01570afa-0030-4a73-ae58-17852df031ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '612875c8fc7240e5b18fe38a034a3ef5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:56.528033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-01570afa-0030-4a73-ae58-17852df031ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '612875c8fc7240e5b18fe38a034a3ef5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:56.528652 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-01570afa-0030-4a73-ae58-17852df031ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '612875c8fc7240e5b18fe38a034a3ef5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:56.529255 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-01570afa-0030-4a73-ae58-17852df031ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '612875c8fc7240e5b18fe38a034a3ef5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:56.543501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-01570afa-0030-4a73-ae58-17852df031ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '612875c8fc7240e5b18fe38a034a3ef5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:56.556277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1348/2687] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:12:56 2022] DELETE /identity/v3/users/612875c8fc7240e5b18fe38a034a3ef5 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:12:56.799000 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a49a4-f13b-422f-9b11-09f3a18a5b88 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:56.809628 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404a49a4-f13b-422f-9b11-09f3a18a5b88 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:56.872388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28a4376-14cb-47bc-b2d6-b83256a786d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:56.873068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a4376-14cb-47bc-b2d6-b83256a786d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:56.873351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a4376-14cb-47bc-b2d6-b83256a786d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:56.873621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28a4376-14cb-47bc-b2d6-b83256a786d0 admin admin] PATH_INFO: `/v3/projects/f6595854cfb04a11994c6aed8f5857c6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:56.874769 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a4376-14cb-47bc-b2d6-b83256a786d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6595854cfb04a11994c6aed8f5857c6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:56.876732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28a4376-14cb-47bc-b2d6-b83256a786d0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:56.890170 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:12:57.021834 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c28a4376-14cb-47bc-b2d6-b83256a786d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6595854cfb04a11994c6aed8f5857c6', 'request_id': 'req-c28a4376-14cb-47bc-b2d6-b83256a786d0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:57.028085 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c28a4376-14cb-47bc-b2d6-b83256a786d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6595854cfb04a11994c6aed8f5857c6', 'request_id': 'req-c28a4376-14cb-47bc-b2d6-b83256a786d0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:57.129261 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c28a4376-14cb-47bc-b2d6-b83256a786d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6595854cfb04a11994c6aed8f5857c6', 'request_id': 'req-c28a4376-14cb-47bc-b2d6-b83256a786d0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:57.140354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c28a4376-14cb-47bc-b2d6-b83256a786d0 admin admin] The token cache is being invalidate because project f6595854cfb04a11994c6aed8f5857c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:12:57.140638 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c28a4376-14cb-47bc-b2d6-b83256a786d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:57.141432 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c28a4376-14cb-47bc-b2d6-b83256a786d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:12:57.143085 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1340/2688] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:12:56 2022] DELETE /identity/v3/projects/f6595854cfb04a11994c6aed8f5857c6 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:12:57.148863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906064f-0da5-48d4-9ef0-b76d70f33832 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:57.149149 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906064f-0da5-48d4-9ef0-b76d70f33832 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:57.149429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d906064f-0da5-48d4-9ef0-b76d70f33832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:57.169944 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d906064f-0da5-48d4-9ef0-b76d70f33832 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:12:57.171178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d906064f-0da5-48d4-9ef0-b76d70f33832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:57.231820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1349/2689] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:12:57.241528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a7a3d-2247-4f6d-9056-bcc90741d1c7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:57.241810 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a7a3d-2247-4f6d-9056-bcc90741d1c7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:57.242087 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021a7a3d-2247-4f6d-9056-bcc90741d1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:57.263029 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-021a7a3d-2247-4f6d-9056-bcc90741d1c7 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:12:57.264278 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-021a7a3d-2247-4f6d-9056-bcc90741d1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:57.275053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1341/2690] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:12:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:12:57.279488 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906064f-0da5-48d4-9ef0-b76d70f33832 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:57.289394 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d906064f-0da5-48d4-9ef0-b76d70f33832 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:57.294071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2224c-24dc-4ee6-a483-cf87bcc62d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:57.294712 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2224c-24dc-4ee6-a483-cf87bcc62d2a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:57.294998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2224c-24dc-4ee6-a483-cf87bcc62d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:57.295278 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d2224c-24dc-4ee6-a483-cf87bcc62d2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:57.295991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2224c-24dc-4ee6-a483-cf87bcc62d2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:12:57.296299 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2224c-24dc-4ee6-a483-cf87bcc62d2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:57.297925 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d2224c-24dc-4ee6-a483-cf87bcc62d2a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:57.310573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1350/2691] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:12:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:12:57.314943 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a7a3d-2247-4f6d-9056-bcc90741d1c7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:57.325524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021a7a3d-2247-4f6d-9056-bcc90741d1c7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:57.330306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784f151-f1ba-4a28-8237-f3ba03137515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:57.330906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784f151-f1ba-4a28-8237-f3ba03137515 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:57.331188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784f151-f1ba-4a28-8237-f3ba03137515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:57.331518 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3784f151-f1ba-4a28-8237-f3ba03137515 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:57.332214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784f151-f1ba-4a28-8237-f3ba03137515 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:57.334110 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3784f151-f1ba-4a28-8237-f3ba03137515 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:57.438889 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1342/2692] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:12:57 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:12:57.443980 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2224c-24dc-4ee6-a483-cf87bcc62d2a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:57.454048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d2224c-24dc-4ee6-a483-cf87bcc62d2a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:57.511923 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368bd589-1237-4ded-8bbc-db572e7f2260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:57.512564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bd589-1237-4ded-8bbc-db572e7f2260 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:57.512851 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bd589-1237-4ded-8bbc-db572e7f2260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:57.513126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368bd589-1237-4ded-8bbc-db572e7f2260 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:57.513769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368bd589-1237-4ded-8bbc-db572e7f2260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:57.515729 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368bd589-1237-4ded-8bbc-db572e7f2260 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:57.524051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:12:57.615208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1351/2693] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:12:57 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:12:57.619347 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784f151-f1ba-4a28-8237-f3ba03137515 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:57.630396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3784f151-f1ba-4a28-8237-f3ba03137515 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:57.634951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043e529-c73f-4129-8d09-9033eff0f3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:57.635550 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043e529-c73f-4129-8d09-9033eff0f3e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:57.635865 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043e529-c73f-4129-8d09-9033eff0f3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:57.636137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043e529-c73f-4129-8d09-9033eff0f3e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:57.636869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043e529-c73f-4129-8d09-9033eff0f3e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:57.638451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a043e529-c73f-4129-8d09-9033eff0f3e8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:57.649645 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1343/2694] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:12:57.656652 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368bd589-1237-4ded-8bbc-db572e7f2260 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:57.667392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368bd589-1237-4ded-8bbc-db572e7f2260 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:57.672202 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba896fef-d03e-4690-be1f-1e788ee2fd5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:57.672812 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba896fef-d03e-4690-be1f-1e788ee2fd5a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:57.673081 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba896fef-d03e-4690-be1f-1e788ee2fd5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:57.673352 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba896fef-d03e-4690-be1f-1e788ee2fd5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:57.673984 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba896fef-d03e-4690-be1f-1e788ee2fd5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:57.676429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba896fef-d03e-4690-be1f-1e788ee2fd5a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:57.687516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1352/2695] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:12:57.696801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043e529-c73f-4129-8d09-9033eff0f3e8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:57.707001 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043e529-c73f-4129-8d09-9033eff0f3e8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:57.711929 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe75f4-64b8-4341-b7a4-c754d8c2a709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:57.712585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe75f4-64b8-4341-b7a4-c754d8c2a709 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:57.712863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe75f4-64b8-4341-b7a4-c754d8c2a709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:57.713141 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbe75f4-64b8-4341-b7a4-c754d8c2a709 admin admin] PATH_INFO: `/v3/projects/66c83d0e9a3a43e2a0a7e9a51315c9f6/users/87221b278f764a4e83148733be19106c/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:57.729796 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe75f4-64b8-4341-b7a4-c754d8c2a709 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c83d0e9a3a43e2a0a7e9a51315c9f6, user_id=87221b278f764a4e83148733be19106c, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:57.732516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbe75f4-64b8-4341-b7a4-c754d8c2a709 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:57.796923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1344/2696] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:12:57 2022] PUT /identity/v3/projects/66c83d0e9a3a43e2a0a7e9a51315c9f6/users/87221b278f764a4e83148733be19106c/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:12:57.808138 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b846b-fc25-4092-bd92-d9dc3a0ec3a4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:57.814315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b846b-fc25-4092-bd92-d9dc3a0ec3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:57.814315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7b846b-fc25-4092-bd92-d9dc3a0ec3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:57.837137 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ad7b846b-fc25-4092-bd92-d9dc3a0ec3a4 None None] MFA Rules not processed for user `87221b278f764a4e83148733be19106c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:12:57.838350 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ad7b846b-fc25-4092-bd92-d9dc3a0ec3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:57.906796 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1353/2697] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:12:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:12:57.919088 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe75f4-64b8-4341-b7a4-c754d8c2a709 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:12:57.920731 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1bbe75f4-64b8-4341-b7a4-c754d8c2a709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:12:58.002899 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbe75f4-64b8-4341-b7a4-c754d8c2a709 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:12:58.008811 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7ff536-b433-4420-a2f0-17f2376a9df3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:12:58.012824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ff536-b433-4420-a2f0-17f2376a9df3 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:12:58.012824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ff536-b433-4420-a2f0-17f2376a9df3 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:12:58.012824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7ff536-b433-4420-a2f0-17f2376a9df3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:12:58.017795 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ff536-b433-4420-a2f0-17f2376a9df3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:12:58.020231 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7ff536-b433-4420-a2f0-17f2376a9df3 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:12:58.029323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1345/2698] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:12:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:13:02.343544 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b846b-fc25-4092-bd92-d9dc3a0ec3a4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:02.412665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7b846b-fc25-4092-bd92-d9dc3a0ec3a4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:02.417104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219fbc3-0c7a-4446-9a3a-039b1ddd7282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:02.417733 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219fbc3-0c7a-4446-9a3a-039b1ddd7282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:02.418013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219fbc3-0c7a-4446-9a3a-039b1ddd7282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:02.418317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4219fbc3-0c7a-4446-9a3a-039b1ddd7282 admin admin] PATH_INFO: `/v3/users/87221b278f764a4e83148733be19106c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:02.419431 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4219fbc3-0c7a-4446-9a3a-039b1ddd7282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87221b278f764a4e83148733be19106c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:02.421404 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4219fbc3-0c7a-4446-9a3a-039b1ddd7282 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:02.714481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4219fbc3-0c7a-4446-9a3a-039b1ddd7282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87221b278f764a4e83148733be19106c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:02.825651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4219fbc3-0c7a-4446-9a3a-039b1ddd7282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87221b278f764a4e83148733be19106c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:02.826285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4219fbc3-0c7a-4446-9a3a-039b1ddd7282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87221b278f764a4e83148733be19106c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:02.826900 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4219fbc3-0c7a-4446-9a3a-039b1ddd7282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87221b278f764a4e83148733be19106c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:02.839606 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:02.841048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4219fbc3-0c7a-4446-9a3a-039b1ddd7282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87221b278f764a4e83148733be19106c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:02.854176 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1354/2699] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:13:02 2022] DELETE /identity/v3/users/87221b278f764a4e83148733be19106c => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:02.858701 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615abdf-ca43-4591-b057-96c62650a3d3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:02.859199 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615abdf-ca43-4591-b057-96c62650a3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:02.859308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a615abdf-ca43-4591-b057-96c62650a3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:02.879973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a615abdf-ca43-4591-b057-96c62650a3d3 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:02.881304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a615abdf-ca43-4591-b057-96c62650a3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:02.893286 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:02.943860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1346/2700] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:02.954024 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219fbc3-0c7a-4446-9a3a-039b1ddd7282 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:02.955469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4219fbc3-0c7a-4446-9a3a-039b1ddd7282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:03.015104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4219fbc3-0c7a-4446-9a3a-039b1ddd7282 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:03.019412 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448efe23-b3cd-4bb4-a874-de8409c158cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:03.020198 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448efe23-b3cd-4bb4-a874-de8409c158cd service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:03.020472 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448efe23-b3cd-4bb4-a874-de8409c158cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:03.020743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448efe23-b3cd-4bb4-a874-de8409c158cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:03.030370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448efe23-b3cd-4bb4-a874-de8409c158cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:03.031832 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448efe23-b3cd-4bb4-a874-de8409c158cd service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:03.041727 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1355/2701] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:13:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:13:03.532069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615abdf-ca43-4591-b057-96c62650a3d3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:03.534520 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a615abdf-ca43-4591-b057-96c62650a3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:03.550855 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a615abdf-ca43-4591-b057-96c62650a3d3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:03.557539 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eedf745-2145-4915-8803-dd81ca987e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:03.558156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedf745-2145-4915-8803-dd81ca987e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:03.558437 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedf745-2145-4915-8803-dd81ca987e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:03.558688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eedf745-2145-4915-8803-dd81ca987e6a admin admin] PATH_INFO: `/v3/projects/66c83d0e9a3a43e2a0a7e9a51315c9f6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:03.559864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedf745-2145-4915-8803-dd81ca987e6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=66c83d0e9a3a43e2a0a7e9a51315c9f6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:03.561786 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eedf745-2145-4915-8803-dd81ca987e6a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:03.701106 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7eedf745-2145-4915-8803-dd81ca987e6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66c83d0e9a3a43e2a0a7e9a51315c9f6', 'request_id': 'req-7eedf745-2145-4915-8803-dd81ca987e6a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:03.707913 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7eedf745-2145-4915-8803-dd81ca987e6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66c83d0e9a3a43e2a0a7e9a51315c9f6', 'request_id': 'req-7eedf745-2145-4915-8803-dd81ca987e6a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:03.780106 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7eedf745-2145-4915-8803-dd81ca987e6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66c83d0e9a3a43e2a0a7e9a51315c9f6', 'request_id': 'req-7eedf745-2145-4915-8803-dd81ca987e6a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:03.792531 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7eedf745-2145-4915-8803-dd81ca987e6a admin admin] The token cache is being invalidate because project 66c83d0e9a3a43e2a0a7e9a51315c9f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:13:03.792823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7eedf745-2145-4915-8803-dd81ca987e6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:03.793479 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7eedf745-2145-4915-8803-dd81ca987e6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:03.795065 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1347/2702] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:13:03 2022] DELETE /identity/v3/projects/66c83d0e9a3a43e2a0a7e9a51315c9f6 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:03.801088 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9ab89-c32c-4ff2-9875-4be1fcbfdc36 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:03.801788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9ab89-c32c-4ff2-9875-4be1fcbfdc36 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:03.801788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a9ab89-c32c-4ff2-9875-4be1fcbfdc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:03.822404 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-97a9ab89-c32c-4ff2-9875-4be1fcbfdc36 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:03.823684 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-97a9ab89-c32c-4ff2-9875-4be1fcbfdc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:03.886746 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1356/2703] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:03.895640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508445c-beaf-4f54-a83e-43f998ee8e06 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:03.896185 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508445c-beaf-4f54-a83e-43f998ee8e06 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:03.896295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508445c-beaf-4f54-a83e-43f998ee8e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:03.917545 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e508445c-beaf-4f54-a83e-43f998ee8e06 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:03.918858 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e508445c-beaf-4f54-a83e-43f998ee8e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:03.928307 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1348/2704] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:03.932651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9ab89-c32c-4ff2-9875-4be1fcbfdc36 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:03.943093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a9ab89-c32c-4ff2-9875-4be1fcbfdc36 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:03.948156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507f76c-93f5-4baa-9779-fbe9b4630786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:03.948766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507f76c-93f5-4baa-9779-fbe9b4630786 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:03.949044 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507f76c-93f5-4baa-9779-fbe9b4630786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:03.949318 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507f76c-93f5-4baa-9779-fbe9b4630786 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:03.949960 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507f76c-93f5-4baa-9779-fbe9b4630786 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:13:03.950258 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507f76c-93f5-4baa-9779-fbe9b4630786 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:03.951968 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507f76c-93f5-4baa-9779-fbe9b4630786 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:03.964956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1357/2705] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:13:03.973049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508445c-beaf-4f54-a83e-43f998ee8e06 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:03.983131 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508445c-beaf-4f54-a83e-43f998ee8e06 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:03.988931 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad6f45-be1f-4d19-a4e8-b155a125c4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:03.989938 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad6f45-be1f-4d19-a4e8-b155a125c4c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:03.989938 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad6f45-be1f-4d19-a4e8-b155a125c4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:03.993068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edad6f45-be1f-4d19-a4e8-b155a125c4c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:03.993068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad6f45-be1f-4d19-a4e8-b155a125c4c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:03.993068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edad6f45-be1f-4d19-a4e8-b155a125c4c5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:04.109796 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1349/2706] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:13:03 2022] POST /identity/v3/projects => generated 371 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:13:04.114147 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507f76c-93f5-4baa-9779-fbe9b4630786 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:04.124625 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507f76c-93f5-4baa-9779-fbe9b4630786 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:04.184866 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f23c6-5ae0-4d0b-9d34-c66c925b1489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:04.185478 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f23c6-5ae0-4d0b-9d34-c66c925b1489 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:04.185759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f23c6-5ae0-4d0b-9d34-c66c925b1489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:04.186036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5f23c6-5ae0-4d0b-9d34-c66c925b1489 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:04.186665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f23c6-5ae0-4d0b-9d34-c66c925b1489 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:04.188615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5f23c6-5ae0-4d0b-9d34-c66c925b1489 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:04.357914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1358/2707] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:13:04 2022] POST /identity/v3/users => generated 286 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:13:04.361638 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad6f45-be1f-4d19-a4e8-b155a125c4c5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:04.372386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edad6f45-be1f-4d19-a4e8-b155a125c4c5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:04.376929 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280a990-b689-4ba8-bc93-23b8353bcf18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:04.377526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280a990-b689-4ba8-bc93-23b8353bcf18 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:04.377841 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280a990-b689-4ba8-bc93-23b8353bcf18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:04.378666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280a990-b689-4ba8-bc93-23b8353bcf18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:04.379772 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280a990-b689-4ba8-bc93-23b8353bcf18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:04.380973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280a990-b689-4ba8-bc93-23b8353bcf18 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:04.391340 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1350/2708] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:13:04.396382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f23c6-5ae0-4d0b-9d34-c66c925b1489 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:04.407282 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5f23c6-5ae0-4d0b-9d34-c66c925b1489 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:04.412760 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e3c99-f033-4412-9391-d30f55d5296c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:04.415469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e3c99-f033-4412-9391-d30f55d5296c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:04.415469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e3c99-f033-4412-9391-d30f55d5296c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:04.415469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e3c99-f033-4412-9391-d30f55d5296c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:04.415469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e3c99-f033-4412-9391-d30f55d5296c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:04.416496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e3c99-f033-4412-9391-d30f55d5296c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:04.418220 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:04.429259 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1359/2709] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:13:04.437103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280a990-b689-4ba8-bc93-23b8353bcf18 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:04.448536 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280a990-b689-4ba8-bc93-23b8353bcf18 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:04.453304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a70c21-098b-4401-9e9f-77b2c8359430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:04.459768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a70c21-098b-4401-9e9f-77b2c8359430 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:04.459768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a70c21-098b-4401-9e9f-77b2c8359430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:04.459768 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a70c21-098b-4401-9e9f-77b2c8359430 admin admin] PATH_INFO: `/v3/projects/f6bf1723e8244af083a35fab6198e2b5/users/2e79cfacd070475191bbd0f2c6090fa8/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:04.470558 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a70c21-098b-4401-9e9f-77b2c8359430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6bf1723e8244af083a35fab6198e2b5, user_id=2e79cfacd070475191bbd0f2c6090fa8, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:04.473745 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a70c21-098b-4401-9e9f-77b2c8359430 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:04.579788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1351/2710] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:13:04 2022] PUT /identity/v3/projects/f6bf1723e8244af083a35fab6198e2b5/users/2e79cfacd070475191bbd0f2c6090fa8/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:04.585633 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01191857-bfbb-45f1-aed4-ea56a85fa347 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:04.585942 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01191857-bfbb-45f1-aed4-ea56a85fa347 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:04.586214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01191857-bfbb-45f1-aed4-ea56a85fa347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:04.604971 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-01191857-bfbb-45f1-aed4-ea56a85fa347 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:04.606213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-01191857-bfbb-45f1-aed4-ea56a85fa347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:04.665431 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1360/2711] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:04.676116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a70c21-098b-4401-9e9f-77b2c8359430 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:04.677970 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-40a70c21-098b-4401-9e9f-77b2c8359430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:04.721997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:04.745802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a70c21-098b-4401-9e9f-77b2c8359430 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:04.745802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611c313-2f52-4dad-8989-b8d28c4c76d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:04.745802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611c313-2f52-4dad-8989-b8d28c4c76d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:04.745802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611c313-2f52-4dad-8989-b8d28c4c76d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:04.746198 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9611c313-2f52-4dad-8989-b8d28c4c76d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:04.759476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611c313-2f52-4dad-8989-b8d28c4c76d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:04.761112 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9611c313-2f52-4dad-8989-b8d28c4c76d3 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:04.774434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1352/2712] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:13:06.558848 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825050b-f5d0-4d25-9eef-b34ee83eba8f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:06.559395 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825050b-f5d0-4d25-9eef-b34ee83eba8f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:06.561722 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825050b-f5d0-4d25-9eef-b34ee83eba8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:06.590649 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2825050b-f5d0-4d25-9eef-b34ee83eba8f None None] MFA Rules not processed for user `2e79cfacd070475191bbd0f2c6090fa8`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:06.591903 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2825050b-f5d0-4d25-9eef-b34ee83eba8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:06.659050 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1361/2713] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:06.663794 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611c313-2f52-4dad-8989-b8d28c4c76d3 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:06.673804 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9611c313-2f52-4dad-8989-b8d28c4c76d3 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:06.678756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3511774-b1c2-4035-93ac-1b97cbc6054b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:06.679398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3511774-b1c2-4035-93ac-1b97cbc6054b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:06.679746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3511774-b1c2-4035-93ac-1b97cbc6054b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:06.680026 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3511774-b1c2-4035-93ac-1b97cbc6054b admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:06.680660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3511774-b1c2-4035-93ac-1b97cbc6054b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:06.682743 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3511774-b1c2-4035-93ac-1b97cbc6054b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:06.788082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1353/2714] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:13:06 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:13:06.792255 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825050b-f5d0-4d25-9eef-b34ee83eba8f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:06.804256 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825050b-f5d0-4d25-9eef-b34ee83eba8f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:06.878933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3bb2c-18a0-4e13-a26c-2a3bfd6b8abc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:06.879611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3bb2c-18a0-4e13-a26c-2a3bfd6b8abc admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:06.879919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3bb2c-18a0-4e13-a26c-2a3bfd6b8abc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:06.880187 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d3bb2c-18a0-4e13-a26c-2a3bfd6b8abc admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:06.880843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3bb2c-18a0-4e13-a26c-2a3bfd6b8abc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:06.882739 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d3bb2c-18a0-4e13-a26c-2a3bfd6b8abc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:07.003158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1362/2715] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:13:06 2022] POST /identity/v3/users => generated 293 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:13:07.007719 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3511774-b1c2-4035-93ac-1b97cbc6054b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:07.017986 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3511774-b1c2-4035-93ac-1b97cbc6054b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:07.023240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39976da6-d838-4b60-b255-381a37b72379 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:07.023890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39976da6-d838-4b60-b255-381a37b72379 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:07.024176 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39976da6-d838-4b60-b255-381a37b72379 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:07.024451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39976da6-d838-4b60-b255-381a37b72379 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:07.025103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39976da6-d838-4b60-b255-381a37b72379 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:07.026714 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39976da6-d838-4b60-b255-381a37b72379 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:07.037290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1354/2716] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:13:07.041384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3bb2c-18a0-4e13-a26c-2a3bfd6b8abc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:07.052174 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d3bb2c-18a0-4e13-a26c-2a3bfd6b8abc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:07.058597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd4c4e-a101-4d70-813d-0b0b54fa6a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:07.058597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd4c4e-a101-4d70-813d-0b0b54fa6a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:07.058597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd4c4e-a101-4d70-813d-0b0b54fa6a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:07.058597 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd4c4e-a101-4d70-813d-0b0b54fa6a73 admin admin] PATH_INFO: `/v3/projects/455353a09f254c8c8c419ba30ede876e/users/bbe1d22bc9e74a9c880189d669715d82/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:07.071990 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:07.075117 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd4c4e-a101-4d70-813d-0b0b54fa6a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=455353a09f254c8c8c419ba30ede876e, user_id=bbe1d22bc9e74a9c880189d669715d82, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:07.078114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd4c4e-a101-4d70-813d-0b0b54fa6a73 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:07.168664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1363/2717] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:13:07 2022] PUT /identity/v3/projects/455353a09f254c8c8c419ba30ede876e/users/bbe1d22bc9e74a9c880189d669715d82/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:13:07.173464 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39976da6-d838-4b60-b255-381a37b72379 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:07.183886 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39976da6-d838-4b60-b255-381a37b72379 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:07.242906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b5f632-7b6c-43df-a578-c9261b8380b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:07.243516 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5f632-7b6c-43df-a578-c9261b8380b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:07.243837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5f632-7b6c-43df-a578-c9261b8380b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:07.244116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b5f632-7b6c-43df-a578-c9261b8380b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:07.244766 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5f632-7b6c-43df-a578-c9261b8380b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:07.246383 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b5f632-7b6c-43df-a578-c9261b8380b0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:07.258108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1355/2718] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:13:07.262345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd4c4e-a101-4d70-813d-0b0b54fa6a73 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:07.273291 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd4c4e-a101-4d70-813d-0b0b54fa6a73 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:07.278076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af3ebc-a259-4343-8fc3-7f8f23a272d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:07.278731 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af3ebc-a259-4343-8fc3-7f8f23a272d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:07.279012 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af3ebc-a259-4343-8fc3-7f8f23a272d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:07.279283 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af3ebc-a259-4343-8fc3-7f8f23a272d3 admin admin] PATH_INFO: `/v3/projects/455353a09f254c8c8c419ba30ede876e/users/bbe1d22bc9e74a9c880189d669715d82/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:07.281257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af3ebc-a259-4343-8fc3-7f8f23a272d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=455353a09f254c8c8c419ba30ede876e, user_id=bbe1d22bc9e74a9c880189d669715d82, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:07.283906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35af3ebc-a259-4343-8fc3-7f8f23a272d3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:07.295196 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1364/2719] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:13:07 2022] PUT /identity/v3/projects/455353a09f254c8c8c419ba30ede876e/users/bbe1d22bc9e74a9c880189d669715d82/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:08.734674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ae1c9-232e-4f6f-bae6-9c707909bec7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:08.735248 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ae1c9-232e-4f6f-bae6-9c707909bec7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:08.735248 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ae1c9-232e-4f6f-bae6-9c707909bec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:08.765982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-064ae1c9-232e-4f6f-bae6-9c707909bec7 None None] MFA Rules not processed for user `bbe1d22bc9e74a9c880189d669715d82`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:08.767320 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-064ae1c9-232e-4f6f-bae6-9c707909bec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:08.843317 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1356/2720] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:08.852588 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af3ebc-a259-4343-8fc3-7f8f23a272d3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:08.854519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-35af3ebc-a259-4343-8fc3-7f8f23a272d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:08.918906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af3ebc-a259-4343-8fc3-7f8f23a272d3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:08.922550 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5489213-c98a-485f-bca2-69d1039d24d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:08.923151 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5489213-c98a-485f-bca2-69d1039d24d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:08.923432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5489213-c98a-485f-bca2-69d1039d24d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:08.923817 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5489213-c98a-485f-bca2-69d1039d24d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:08.986076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5489213-c98a-485f-bca2-69d1039d24d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:08.988098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5489213-c98a-485f-bca2-69d1039d24d4 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:08.998385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1365/2721] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:13:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:13:10.826509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ae1c9-232e-4f6f-bae6-9c707909bec7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:10.828344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-064ae1c9-232e-4f6f-bae6-9c707909bec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:10.844480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ae1c9-232e-4f6f-bae6-9c707909bec7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:10.848480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e086e66-6ee0-4956-930b-0dae98db7655 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:10.849084 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e086e66-6ee0-4956-930b-0dae98db7655 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:10.849363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e086e66-6ee0-4956-930b-0dae98db7655 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:10.849633 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e086e66-6ee0-4956-930b-0dae98db7655 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:10.858808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e086e66-6ee0-4956-930b-0dae98db7655 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:10.860251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e086e66-6ee0-4956-930b-0dae98db7655 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:10.870229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1357/2722] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:13:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:13:18.993961 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:20.866014 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:22.008338 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de479b5-61de-48e3-a3a6-3c38d66cbcc7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:22.008895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de479b5-61de-48e3-a3a6-3c38d66cbcc7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:22.008895 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de479b5-61de-48e3-a3a6-3c38d66cbcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:22.032982 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4de479b5-61de-48e3-a3a6-3c38d66cbcc7 None None] MFA Rules not processed for user `ecf12b3f54e349d5a8f6587410c79501`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:22.035116 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4de479b5-61de-48e3-a3a6-3c38d66cbcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:22.105350 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1366/2723] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:22.142974 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e086e66-6ee0-4956-930b-0dae98db7655 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:22.145972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3e086e66-6ee0-4956-930b-0dae98db7655 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:22.222179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e086e66-6ee0-4956-930b-0dae98db7655 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:22.225962 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9e5df-873e-4c8e-95c7-30302bdf453f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6dee005fc3d4fb593a1771e7851c20a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:22.226590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9e5df-873e-4c8e-95c7-30302bdf453f service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:22.226870 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9e5df-873e-4c8e-95c7-30302bdf453f service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:22.227144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a9e5df-873e-4c8e-95c7-30302bdf453f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:22.229088 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e2a9e5df-873e-4c8e-95c7-30302bdf453f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:22.321598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9e5df-873e-4c8e-95c7-30302bdf453f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:22.323240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a9e5df-873e-4c8e-95c7-30302bdf453f service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:22.336681 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1358/2724] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:13:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:13:23.119343 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de479b5-61de-48e3-a3a6-3c38d66cbcc7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:23.207564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de479b5-61de-48e3-a3a6-3c38d66cbcc7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:23.213287 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee848c17-b555-4b86-9f55-1ee73b06169e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:23.213952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee848c17-b555-4b86-9f55-1ee73b06169e service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:23.215686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee848c17-b555-4b86-9f55-1ee73b06169e service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:23.215686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee848c17-b555-4b86-9f55-1ee73b06169e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:23.219186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ee848c17-b555-4b86-9f55-1ee73b06169e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:23.235962 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee848c17-b555-4b86-9f55-1ee73b06169e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:23.237864 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee848c17-b555-4b86-9f55-1ee73b06169e service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:23.251906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1367/2725] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:13:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:13:32.089808 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9e5df-873e-4c8e-95c7-30302bdf453f service glance] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:32.163887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a9e5df-873e-4c8e-95c7-30302bdf453f service glance] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:32.184232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704bad4-168b-4a3a-a872-512b1891d6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:32.184232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704bad4-168b-4a3a-a872-512b1891d6e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:32.184232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704bad4-168b-4a3a-a872-512b1891d6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:32.184232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d704bad4-168b-4a3a-a872-512b1891d6e1 admin admin] PATH_INFO: `/v3/users/2e79cfacd070475191bbd0f2c6090fa8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:32.184705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704bad4-168b-4a3a-a872-512b1891d6e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e79cfacd070475191bbd0f2c6090fa8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:32.184705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d704bad4-168b-4a3a-a872-512b1891d6e1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:32.392489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:32.396348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d704bad4-168b-4a3a-a872-512b1891d6e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e79cfacd070475191bbd0f2c6090fa8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:32.505788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d704bad4-168b-4a3a-a872-512b1891d6e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e79cfacd070475191bbd0f2c6090fa8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:32.506618 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d704bad4-168b-4a3a-a872-512b1891d6e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e79cfacd070475191bbd0f2c6090fa8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:32.507334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d704bad4-168b-4a3a-a872-512b1891d6e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e79cfacd070475191bbd0f2c6090fa8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:32.515272 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d704bad4-168b-4a3a-a872-512b1891d6e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e79cfacd070475191bbd0f2c6090fa8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:32.534946 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1359/2726] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:13:32 2022] DELETE /identity/v3/users/2e79cfacd070475191bbd0f2c6090fa8 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:32.855794 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee848c17-b555-4b86-9f55-1ee73b06169e service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:32.868968 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee848c17-b555-4b86-9f55-1ee73b06169e service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:32.930257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7108f2-957b-4013-893a-124f9bf00484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:32.930870 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7108f2-957b-4013-893a-124f9bf00484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:32.931150 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7108f2-957b-4013-893a-124f9bf00484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:32.931419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7108f2-957b-4013-893a-124f9bf00484 admin admin] PATH_INFO: `/v3/projects/f6bf1723e8244af083a35fab6198e2b5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:32.932603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7108f2-957b-4013-893a-124f9bf00484 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6bf1723e8244af083a35fab6198e2b5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:32.934481 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7108f2-957b-4013-893a-124f9bf00484 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:33.078142 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5f7108f2-957b-4013-893a-124f9bf00484 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6bf1723e8244af083a35fab6198e2b5', 'request_id': 'req-5f7108f2-957b-4013-893a-124f9bf00484'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:33.079158 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:33.085610 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5f7108f2-957b-4013-893a-124f9bf00484 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6bf1723e8244af083a35fab6198e2b5', 'request_id': 'req-5f7108f2-957b-4013-893a-124f9bf00484'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:33.219409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5f7108f2-957b-4013-893a-124f9bf00484 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6bf1723e8244af083a35fab6198e2b5', 'request_id': 'req-5f7108f2-957b-4013-893a-124f9bf00484'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:33.230880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5f7108f2-957b-4013-893a-124f9bf00484 admin admin] The token cache is being invalidate because project f6bf1723e8244af083a35fab6198e2b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:13:33.231172 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5f7108f2-957b-4013-893a-124f9bf00484 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:33.231840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5f7108f2-957b-4013-893a-124f9bf00484 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:33.233332 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1368/2727] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:13:32 2022] DELETE /identity/v3/projects/f6bf1723e8244af083a35fab6198e2b5 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:33.238139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704bad4-168b-4a3a-a872-512b1891d6e1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:33.239689 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d704bad4-168b-4a3a-a872-512b1891d6e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:33.253715 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d704bad4-168b-4a3a-a872-512b1891d6e1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:33.311628 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c809d-7694-42ef-bab0-4b862a035fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:33.312296 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c809d-7694-42ef-bab0-4b862a035fbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:33.312576 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c809d-7694-42ef-bab0-4b862a035fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:33.312848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c809d-7694-42ef-bab0-4b862a035fbb admin admin] PATH_INFO: `/v3/users/bbe1d22bc9e74a9c880189d669715d82` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:33.314017 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c809d-7694-42ef-bab0-4b862a035fbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbe1d22bc9e74a9c880189d669715d82)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:33.316730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c809d-7694-42ef-bab0-4b862a035fbb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:33.535961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7d5c809d-7694-42ef-bab0-4b862a035fbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbe1d22bc9e74a9c880189d669715d82'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:33.636365 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7d5c809d-7694-42ef-bab0-4b862a035fbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbe1d22bc9e74a9c880189d669715d82'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:33.636984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7d5c809d-7694-42ef-bab0-4b862a035fbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbe1d22bc9e74a9c880189d669715d82'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:33.637682 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7d5c809d-7694-42ef-bab0-4b862a035fbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbe1d22bc9e74a9c880189d669715d82'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:33.646436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7d5c809d-7694-42ef-bab0-4b862a035fbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbe1d22bc9e74a9c880189d669715d82'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:33.664930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1360/2728] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:13:33 2022] DELETE /identity/v3/users/bbe1d22bc9e74a9c880189d669715d82 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:33.930342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7108f2-957b-4013-893a-124f9bf00484 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:33.933913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5f7108f2-957b-4013-893a-124f9bf00484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:33.955541 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7108f2-957b-4013-893a-124f9bf00484 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:34.103573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9e4d96-25e1-449e-a24a-9716f96058c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:34.103573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e4d96-25e1-449e-a24a-9716f96058c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:34.103573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e4d96-25e1-449e-a24a-9716f96058c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:34.103573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9e4d96-25e1-449e-a24a-9716f96058c6 admin admin] PATH_INFO: `/v3/projects/455353a09f254c8c8c419ba30ede876e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:34.104351 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e4d96-25e1-449e-a24a-9716f96058c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=455353a09f254c8c8c419ba30ede876e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:34.104351 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9e4d96-25e1-449e-a24a-9716f96058c6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:34.233329 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2f9e4d96-25e1-449e-a24a-9716f96058c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '455353a09f254c8c8c419ba30ede876e', 'request_id': 'req-2f9e4d96-25e1-449e-a24a-9716f96058c6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:34.239921 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2f9e4d96-25e1-449e-a24a-9716f96058c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '455353a09f254c8c8c419ba30ede876e', 'request_id': 'req-2f9e4d96-25e1-449e-a24a-9716f96058c6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:34.333366 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2f9e4d96-25e1-449e-a24a-9716f96058c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '455353a09f254c8c8c419ba30ede876e', 'request_id': 'req-2f9e4d96-25e1-449e-a24a-9716f96058c6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:34.345311 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2f9e4d96-25e1-449e-a24a-9716f96058c6 admin admin] The token cache is being invalidate because project 455353a09f254c8c8c419ba30ede876e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:13:34.345598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2f9e4d96-25e1-449e-a24a-9716f96058c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:34.346978 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2f9e4d96-25e1-449e-a24a-9716f96058c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:34.348643 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1369/2729] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:13:33 2022] DELETE /identity/v3/projects/455353a09f254c8c8c419ba30ede876e => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:34.353639 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425b67f-4d83-43b9-8e23-f887e2c65534 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:34.354148 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425b67f-4d83-43b9-8e23-f887e2c65534 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:34.354269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d425b67f-4d83-43b9-8e23-f887e2c65534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:34.373803 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d425b67f-4d83-43b9-8e23-f887e2c65534 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:34.375016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d425b67f-4d83-43b9-8e23-f887e2c65534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:34.411583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:34.437036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1361/2730] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:34.446410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d9ba0-77ca-4baf-b725-069f682df62a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:34.446915 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d9ba0-77ca-4baf-b725-069f682df62a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:34.447409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4d9ba0-77ca-4baf-b725-069f682df62a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:34.478560 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5e4d9ba0-77ca-4baf-b725-069f682df62a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:34.479833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5e4d9ba0-77ca-4baf-b725-069f682df62a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:34.488946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1370/2731] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:34.493977 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425b67f-4d83-43b9-8e23-f887e2c65534 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:34.504759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d425b67f-4d83-43b9-8e23-f887e2c65534 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:34.513384 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b22205-5972-42ee-9d22-fb65ba6c88d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:34.514002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b22205-5972-42ee-9d22-fb65ba6c88d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:34.514285 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b22205-5972-42ee-9d22-fb65ba6c88d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:34.514559 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b22205-5972-42ee-9d22-fb65ba6c88d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:34.515211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b22205-5972-42ee-9d22-fb65ba6c88d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:13:34.515515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b22205-5972-42ee-9d22-fb65ba6c88d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:34.517152 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b22205-5972-42ee-9d22-fb65ba6c88d0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:34.536168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1362/2732] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:13:34.540955 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d9ba0-77ca-4baf-b725-069f682df62a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:34.550844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4d9ba0-77ca-4baf-b725-069f682df62a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:34.555672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc0223-7b1e-4f04-b139-0debe5819bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:34.556293 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc0223-7b1e-4f04-b139-0debe5819bee admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:34.556574 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc0223-7b1e-4f04-b139-0debe5819bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:34.556846 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc0223-7b1e-4f04-b139-0debe5819bee admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:34.557486 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc0223-7b1e-4f04-b139-0debe5819bee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:34.559370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc0223-7b1e-4f04-b139-0debe5819bee admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:34.661816 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1371/2733] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:13:34 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:13:34.666007 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b22205-5972-42ee-9d22-fb65ba6c88d0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:34.676846 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b22205-5972-42ee-9d22-fb65ba6c88d0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:34.731846 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95bc02-ce9b-4b15-8bc8-e0ea2e3edb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:34.732533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95bc02-ce9b-4b15-8bc8-e0ea2e3edb16 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:34.732814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95bc02-ce9b-4b15-8bc8-e0ea2e3edb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:34.733084 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc95bc02-ce9b-4b15-8bc8-e0ea2e3edb16 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:34.733702 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95bc02-ce9b-4b15-8bc8-e0ea2e3edb16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:34.735608 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc95bc02-ce9b-4b15-8bc8-e0ea2e3edb16 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:34.843465 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1363/2734] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:13:34 2022] POST /identity/v3/users => generated 292 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:13:34.848675 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc0223-7b1e-4f04-b139-0debe5819bee admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:34.860046 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc0223-7b1e-4f04-b139-0debe5819bee admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:34.865442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39454d72-3dc2-418c-ad7d-5e57b344b246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:34.866137 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39454d72-3dc2-418c-ad7d-5e57b344b246 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:34.866511 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39454d72-3dc2-418c-ad7d-5e57b344b246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:34.866846 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39454d72-3dc2-418c-ad7d-5e57b344b246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:34.867752 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39454d72-3dc2-418c-ad7d-5e57b344b246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:34.869449 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39454d72-3dc2-418c-ad7d-5e57b344b246 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:34.881716 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1372/2735] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:13:34.885732 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95bc02-ce9b-4b15-8bc8-e0ea2e3edb16 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:34.899417 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc95bc02-ce9b-4b15-8bc8-e0ea2e3edb16 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:34.908010 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f58bfc-2201-4191-9561-f10850be5899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:34.908728 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f58bfc-2201-4191-9561-f10850be5899 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:34.909940 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f58bfc-2201-4191-9561-f10850be5899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:34.910279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f58bfc-2201-4191-9561-f10850be5899 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:34.911445 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f58bfc-2201-4191-9561-f10850be5899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:34.914922 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f58bfc-2201-4191-9561-f10850be5899 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:34.928381 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1364/2736] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:13:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:13:34.932615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39454d72-3dc2-418c-ad7d-5e57b344b246 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:34.946002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39454d72-3dc2-418c-ad7d-5e57b344b246 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:34.951371 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63568522-773b-45a8-b85a-d1e0afaf2aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:34.952548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63568522-773b-45a8-b85a-d1e0afaf2aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:34.952548 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63568522-773b-45a8-b85a-d1e0afaf2aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:34.952768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63568522-773b-45a8-b85a-d1e0afaf2aa3 admin admin] PATH_INFO: `/v3/projects/7f5a01116c4e44e7bb69ecf7af8f51c8/users/c1b33abf0e36418d841e7adc3574720d/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:34.968553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63568522-773b-45a8-b85a-d1e0afaf2aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f5a01116c4e44e7bb69ecf7af8f51c8, user_id=c1b33abf0e36418d841e7adc3574720d, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:34.972525 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63568522-773b-45a8-b85a-d1e0afaf2aa3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:35.082861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1373/2737] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:13:34 2022] PUT /identity/v3/projects/7f5a01116c4e44e7bb69ecf7af8f51c8/users/c1b33abf0e36418d841e7adc3574720d/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:35.093401 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65ed51-e38a-4747-9bc0-b4a1daf23abf None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:35.093992 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65ed51-e38a-4747-9bc0-b4a1daf23abf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:35.094455 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65ed51-e38a-4747-9bc0-b4a1daf23abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:35.124875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-aa65ed51-e38a-4747-9bc0-b4a1daf23abf None None] MFA Rules not processed for user `c1b33abf0e36418d841e7adc3574720d`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:35.126147 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-aa65ed51-e38a-4747-9bc0-b4a1daf23abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:35.217098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1365/2738] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:35.231101 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63568522-773b-45a8-b85a-d1e0afaf2aa3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:35.232881 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-63568522-773b-45a8-b85a-d1e0afaf2aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:35.240867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:35.296553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63568522-773b-45a8-b85a-d1e0afaf2aa3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:35.301167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440380ba-178d-40f0-93cb-eedf2c2561b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:35.302324 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440380ba-178d-40f0-93cb-eedf2c2561b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:35.302626 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440380ba-178d-40f0-93cb-eedf2c2561b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:35.302879 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440380ba-178d-40f0-93cb-eedf2c2561b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:35.312427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440380ba-178d-40f0-93cb-eedf2c2561b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:35.313882 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440380ba-178d-40f0-93cb-eedf2c2561b7 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:35.324999 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1374/2739] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:13:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:13:35.345056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65ed51-e38a-4747-9bc0-b4a1daf23abf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:35.388089 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:35.412096 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65ed51-e38a-4747-9bc0-b4a1daf23abf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:35.416827 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c57f0-1205-4b1e-a28d-3a331538a992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:35.417411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c57f0-1205-4b1e-a28d-3a331538a992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:35.417694 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c57f0-1205-4b1e-a28d-3a331538a992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:35.417971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c57f0-1205-4b1e-a28d-3a331538a992 admin admin] PATH_INFO: `/v3/users/c1b33abf0e36418d841e7adc3574720d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:35.419132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c57f0-1205-4b1e-a28d-3a331538a992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1b33abf0e36418d841e7adc3574720d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:35.421120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c57f0-1205-4b1e-a28d-3a331538a992 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:35.658717 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d0c57f0-1205-4b1e-a28d-3a331538a992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1b33abf0e36418d841e7adc3574720d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:35.711579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d0c57f0-1205-4b1e-a28d-3a331538a992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1b33abf0e36418d841e7adc3574720d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:35.712630 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d0c57f0-1205-4b1e-a28d-3a331538a992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1b33abf0e36418d841e7adc3574720d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:35.713299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d0c57f0-1205-4b1e-a28d-3a331538a992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1b33abf0e36418d841e7adc3574720d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:35.721898 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0d0c57f0-1205-4b1e-a28d-3a331538a992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1b33abf0e36418d841e7adc3574720d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:35.741374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1366/2740] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:13:35 2022] DELETE /identity/v3/users/c1b33abf0e36418d841e7adc3574720d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:35.746234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab9877-a61e-4b1f-9f1a-d2beb22707a6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:35.747519 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab9877-a61e-4b1f-9f1a-d2beb22707a6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:35.748296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ab9877-a61e-4b1f-9f1a-d2beb22707a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:35.773749 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-19ab9877-a61e-4b1f-9f1a-d2beb22707a6 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:35.775489 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-19ab9877-a61e-4b1f-9f1a-d2beb22707a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:35.842654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1375/2741] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:35.853030 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c57f0-1205-4b1e-a28d-3a331538a992 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:35.854487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0d0c57f0-1205-4b1e-a28d-3a331538a992 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:35.914782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c57f0-1205-4b1e-a28d-3a331538a992 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:35.918472 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa815394-334c-4d9d-91cf-93dadb7ebdb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:35.919082 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa815394-334c-4d9d-91cf-93dadb7ebdb1 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:35.919371 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa815394-334c-4d9d-91cf-93dadb7ebdb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:35.919665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa815394-334c-4d9d-91cf-93dadb7ebdb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:35.930580 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa815394-334c-4d9d-91cf-93dadb7ebdb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:35.932070 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa815394-334c-4d9d-91cf-93dadb7ebdb1 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:35.942313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1367/2742] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:13:36.501691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab9877-a61e-4b1f-9f1a-d2beb22707a6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:36.503837 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-19ab9877-a61e-4b1f-9f1a-d2beb22707a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:36.520161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ab9877-a61e-4b1f-9f1a-d2beb22707a6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:36.525855 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7cfd07-c0e2-423b-8e68-654345428747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:36.526816 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cfd07-c0e2-423b-8e68-654345428747 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:36.527260 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cfd07-c0e2-423b-8e68-654345428747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:36.527766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7cfd07-c0e2-423b-8e68-654345428747 admin admin] PATH_INFO: `/v3/projects/7f5a01116c4e44e7bb69ecf7af8f51c8` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:36.529884 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cfd07-c0e2-423b-8e68-654345428747 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f5a01116c4e44e7bb69ecf7af8f51c8)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:36.532493 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7cfd07-c0e2-423b-8e68-654345428747 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:36.649367 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6c7cfd07-c0e2-423b-8e68-654345428747 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f5a01116c4e44e7bb69ecf7af8f51c8', 'request_id': 'req-6c7cfd07-c0e2-423b-8e68-654345428747'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:36.655753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6c7cfd07-c0e2-423b-8e68-654345428747 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f5a01116c4e44e7bb69ecf7af8f51c8', 'request_id': 'req-6c7cfd07-c0e2-423b-8e68-654345428747'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:36.716821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6c7cfd07-c0e2-423b-8e68-654345428747 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f5a01116c4e44e7bb69ecf7af8f51c8', 'request_id': 'req-6c7cfd07-c0e2-423b-8e68-654345428747'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:36.735515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6c7cfd07-c0e2-423b-8e68-654345428747 admin admin] The token cache is being invalidate because project 7f5a01116c4e44e7bb69ecf7af8f51c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:13:36.735969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6c7cfd07-c0e2-423b-8e68-654345428747 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:36.736746 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6c7cfd07-c0e2-423b-8e68-654345428747 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:36.738464 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1376/2743] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:13:36 2022] DELETE /identity/v3/projects/7f5a01116c4e44e7bb69ecf7af8f51c8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:36.743605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8b948-1f0a-4d16-abbe-1fe5fa3851cd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:36.744121 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8b948-1f0a-4d16-abbe-1fe5fa3851cd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:36.745110 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd8b948-1f0a-4d16-abbe-1fe5fa3851cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:36.770064 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ffd8b948-1f0a-4d16-abbe-1fe5fa3851cd None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:36.771363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ffd8b948-1f0a-4d16-abbe-1fe5fa3851cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:36.833373 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:36.842291 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1368/2744] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:36.850046 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d31afe-79ea-4c38-b8e8-f852206bd30b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:36.850419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d31afe-79ea-4c38-b8e8-f852206bd30b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:36.850760 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d31afe-79ea-4c38-b8e8-f852206bd30b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:36.879224 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e4d31afe-79ea-4c38-b8e8-f852206bd30b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:36.881058 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e4d31afe-79ea-4c38-b8e8-f852206bd30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:36.890993 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1377/2745] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:13:36.895874 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8b948-1f0a-4d16-abbe-1fe5fa3851cd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:36.906150 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd8b948-1f0a-4d16-abbe-1fe5fa3851cd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:36.916924 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5de33-47e4-49b1-9d09-4f8d5dfee0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:36.917673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5de33-47e4-49b1-9d09-4f8d5dfee0a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:36.918155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5de33-47e4-49b1-9d09-4f8d5dfee0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:36.918482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e5de33-47e4-49b1-9d09-4f8d5dfee0a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:36.919269 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5de33-47e4-49b1-9d09-4f8d5dfee0a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:13:36.919575 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5de33-47e4-49b1-9d09-4f8d5dfee0a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:36.921496 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e5de33-47e4-49b1-9d09-4f8d5dfee0a3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:36.936850 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1369/2746] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:13:36.941581 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d31afe-79ea-4c38-b8e8-f852206bd30b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:36.951430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d31afe-79ea-4c38-b8e8-f852206bd30b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:36.956427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341a8bf-4ea9-4e38-86eb-602a3827022d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:36.957041 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341a8bf-4ea9-4e38-86eb-602a3827022d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:36.957317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341a8bf-4ea9-4e38-86eb-602a3827022d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:36.957603 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8341a8bf-4ea9-4e38-86eb-602a3827022d admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:36.958234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341a8bf-4ea9-4e38-86eb-602a3827022d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:36.960156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8341a8bf-4ea9-4e38-86eb-602a3827022d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:37.082245 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1378/2747] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:13:36 2022] POST /identity/v3/projects => generated 359 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:13:37.085267 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5de33-47e4-49b1-9d09-4f8d5dfee0a3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:37.096960 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e5de33-47e4-49b1-9d09-4f8d5dfee0a3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:37.154656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d366e-4f67-4469-bc74-c4993399fc9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:37.155303 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d366e-4f67-4469-bc74-c4993399fc9f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:37.155585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d366e-4f67-4469-bc74-c4993399fc9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:37.155897 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d366e-4f67-4469-bc74-c4993399fc9f admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:37.156535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d366e-4f67-4469-bc74-c4993399fc9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:37.159208 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d366e-4f67-4469-bc74-c4993399fc9f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:37.300083 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1370/2748] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:13:37 2022] POST /identity/v3/users => generated 280 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:13:37.304656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341a8bf-4ea9-4e38-86eb-602a3827022d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:37.310129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:37.315485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8341a8bf-4ea9-4e38-86eb-602a3827022d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:37.320146 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d997b-d9b0-48c6-953a-a095fa20a816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:37.320808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d997b-d9b0-48c6-953a-a095fa20a816 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:37.321085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d997b-d9b0-48c6-953a-a095fa20a816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:37.321352 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d997b-d9b0-48c6-953a-a095fa20a816 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:37.321983 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d997b-d9b0-48c6-953a-a095fa20a816 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:37.323555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d997b-d9b0-48c6-953a-a095fa20a816 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:37.334740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1379/2749] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:13:37.339233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d366e-4f67-4469-bc74-c4993399fc9f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:37.350124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d366e-4f67-4469-bc74-c4993399fc9f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:37.354952 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dfc772-c425-4609-97d8-2b9e34471231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:37.355558 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfc772-c425-4609-97d8-2b9e34471231 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:37.355873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfc772-c425-4609-97d8-2b9e34471231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:37.356151 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dfc772-c425-4609-97d8-2b9e34471231 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:37.356797 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfc772-c425-4609-97d8-2b9e34471231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:37.358433 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dfc772-c425-4609-97d8-2b9e34471231 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:37.370691 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1371/2750] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:13:37.375544 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d997b-d9b0-48c6-953a-a095fa20a816 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:37.387193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d997b-d9b0-48c6-953a-a095fa20a816 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:37.392280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df4a18-0cf4-412b-afc8-7198b27ffe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:37.393007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df4a18-0cf4-412b-afc8-7198b27ffe49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:37.393804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df4a18-0cf4-412b-afc8-7198b27ffe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:37.394076 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df4a18-0cf4-412b-afc8-7198b27ffe49 admin admin] PATH_INFO: `/v3/projects/a078b71ed5004c5d9dab959b336aeb2a/users/ffbd7ea380eb4cc89f1cb6adebe03ae9/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:37.409526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df4a18-0cf4-412b-afc8-7198b27ffe49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a078b71ed5004c5d9dab959b336aeb2a, user_id=ffbd7ea380eb4cc89f1cb6adebe03ae9, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:37.412279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8df4a18-0cf4-412b-afc8-7198b27ffe49 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:37.501914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1380/2751] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:13:37 2022] PUT /identity/v3/projects/a078b71ed5004c5d9dab959b336aeb2a/users/ffbd7ea380eb4cc89f1cb6adebe03ae9/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:13:37.517116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8ba60-1e1a-4397-b0bf-c8a2b22b5275 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:37.517524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8ba60-1e1a-4397-b0bf-c8a2b22b5275 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:37.519349 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c8ba60-1e1a-4397-b0bf-c8a2b22b5275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:37.549101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f6c8ba60-1e1a-4397-b0bf-c8a2b22b5275 None None] MFA Rules not processed for user `ffbd7ea380eb4cc89f1cb6adebe03ae9`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:37.550456 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f6c8ba60-1e1a-4397-b0bf-c8a2b22b5275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:37.625725 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1372/2752] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:37.677975 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df4a18-0cf4-412b-afc8-7198b27ffe49 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:37.680936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f8df4a18-0cf4-412b-afc8-7198b27ffe49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:37.749821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df4a18-0cf4-412b-afc8-7198b27ffe49 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:37.754387 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6796d6-d9a2-4aaf-93fc-8de586201be6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:37.755002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6796d6-d9a2-4aaf-93fc-8de586201be6 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:37.755282 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6796d6-d9a2-4aaf-93fc-8de586201be6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:37.755552 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6796d6-d9a2-4aaf-93fc-8de586201be6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:37.764783 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6796d6-d9a2-4aaf-93fc-8de586201be6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:37.766350 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6796d6-d9a2-4aaf-93fc-8de586201be6 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:37.776942 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1381/2753] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:13:37 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:13:37.793866 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8ba60-1e1a-4397-b0bf-c8a2b22b5275 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:37.850131 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:37.861006 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c8ba60-1e1a-4397-b0bf-c8a2b22b5275 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:37.866332 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c292eb0-92b3-4fd1-a461-c430573dfb74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:37.867212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c292eb0-92b3-4fd1-a461-c430573dfb74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:37.867600 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c292eb0-92b3-4fd1-a461-c430573dfb74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:37.867969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c292eb0-92b3-4fd1-a461-c430573dfb74 admin admin] PATH_INFO: `/v3/users/ffbd7ea380eb4cc89f1cb6adebe03ae9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:37.869130 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c292eb0-92b3-4fd1-a461-c430573dfb74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffbd7ea380eb4cc89f1cb6adebe03ae9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:37.871681 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c292eb0-92b3-4fd1-a461-c430573dfb74 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:38.193214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4c292eb0-92b3-4fd1-a461-c430573dfb74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffbd7ea380eb4cc89f1cb6adebe03ae9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:38.272027 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4c292eb0-92b3-4fd1-a461-c430573dfb74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffbd7ea380eb4cc89f1cb6adebe03ae9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:38.272645 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4c292eb0-92b3-4fd1-a461-c430573dfb74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffbd7ea380eb4cc89f1cb6adebe03ae9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:38.273247 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4c292eb0-92b3-4fd1-a461-c430573dfb74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffbd7ea380eb4cc89f1cb6adebe03ae9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:38.281109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4c292eb0-92b3-4fd1-a461-c430573dfb74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffbd7ea380eb4cc89f1cb6adebe03ae9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:38.299134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1373/2754] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:13:37 2022] DELETE /identity/v3/users/ffbd7ea380eb4cc89f1cb6adebe03ae9 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:38.304267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4d88b-9e51-4064-87a8-004de08fd291 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:38.310373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4d88b-9e51-4064-87a8-004de08fd291 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:38.310373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb4d88b-9e51-4064-87a8-004de08fd291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:38.324405 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7cb4d88b-9e51-4064-87a8-004de08fd291 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:38.325660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7cb4d88b-9e51-4064-87a8-004de08fd291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:38.387114 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1382/2755] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:38.397265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c292eb0-92b3-4fd1-a461-c430573dfb74 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:38.398660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4c292eb0-92b3-4fd1-a461-c430573dfb74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:38.455304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c292eb0-92b3-4fd1-a461-c430573dfb74 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:38.459253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7680c9-8933-4210-a949-155809fd4b14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:38.459933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7680c9-8933-4210-a949-155809fd4b14 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:38.460214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7680c9-8933-4210-a949-155809fd4b14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:38.460611 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7680c9-8933-4210-a949-155809fd4b14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:38.470499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7680c9-8933-4210-a949-155809fd4b14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:38.472640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7680c9-8933-4210-a949-155809fd4b14 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:38.482481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1374/2756] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:13:38.924208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4d88b-9e51-4064-87a8-004de08fd291 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:38.927407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7cb4d88b-9e51-4064-87a8-004de08fd291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:38.938526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb4d88b-9e51-4064-87a8-004de08fd291 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:38.943677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5709b35-d8c3-4cce-bc01-0e249c0b6d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:38.944343 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5709b35-d8c3-4cce-bc01-0e249c0b6d8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:38.944623 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5709b35-d8c3-4cce-bc01-0e249c0b6d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:38.944893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5709b35-d8c3-4cce-bc01-0e249c0b6d8e admin admin] PATH_INFO: `/v3/projects/a078b71ed5004c5d9dab959b336aeb2a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:38.946018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5709b35-d8c3-4cce-bc01-0e249c0b6d8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a078b71ed5004c5d9dab959b336aeb2a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:38.947942 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5709b35-d8c3-4cce-bc01-0e249c0b6d8e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:39.067740 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a5709b35-d8c3-4cce-bc01-0e249c0b6d8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a078b71ed5004c5d9dab959b336aeb2a', 'request_id': 'req-a5709b35-d8c3-4cce-bc01-0e249c0b6d8e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:39.074617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a5709b35-d8c3-4cce-bc01-0e249c0b6d8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a078b71ed5004c5d9dab959b336aeb2a', 'request_id': 'req-a5709b35-d8c3-4cce-bc01-0e249c0b6d8e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:39.156684 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a5709b35-d8c3-4cce-bc01-0e249c0b6d8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a078b71ed5004c5d9dab959b336aeb2a', 'request_id': 'req-a5709b35-d8c3-4cce-bc01-0e249c0b6d8e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:39.161936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:39.169508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a5709b35-d8c3-4cce-bc01-0e249c0b6d8e admin admin] The token cache is being invalidate because project a078b71ed5004c5d9dab959b336aeb2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:13:39.169762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a5709b35-d8c3-4cce-bc01-0e249c0b6d8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:39.170447 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a5709b35-d8c3-4cce-bc01-0e249c0b6d8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:39.172134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1383/2757] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:13:38 2022] DELETE /identity/v3/projects/a078b71ed5004c5d9dab959b336aeb2a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:39.178396 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef656e-7ae7-434f-a321-a23c8a4400cc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:39.178871 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef656e-7ae7-434f-a321-a23c8a4400cc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:39.178992 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef656e-7ae7-434f-a321-a23c8a4400cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:39.198537 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0cef656e-7ae7-434f-a321-a23c8a4400cc None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:39.199767 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0cef656e-7ae7-434f-a321-a23c8a4400cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:39.263831 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1375/2758] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:39.273152 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a93eb-0052-4167-8253-668d9e17b41a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:39.273441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a93eb-0052-4167-8253-668d9e17b41a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:39.273714 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a93eb-0052-4167-8253-668d9e17b41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:39.294192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a95a93eb-0052-4167-8253-668d9e17b41a None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:39.295416 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a95a93eb-0052-4167-8253-668d9e17b41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:39.303630 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1384/2759] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:39.308392 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef656e-7ae7-434f-a321-a23c8a4400cc None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:39.318143 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef656e-7ae7-434f-a321-a23c8a4400cc None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:39.322996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb4580-805e-4499-801e-f76235a3ec07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:39.323753 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb4580-805e-4499-801e-f76235a3ec07 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:39.324036 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb4580-805e-4499-801e-f76235a3ec07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:39.324306 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cb4580-805e-4499-801e-f76235a3ec07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:39.324946 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb4580-805e-4499-801e-f76235a3ec07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:13:39.325249 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb4580-805e-4499-801e-f76235a3ec07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:39.326973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cb4580-805e-4499-801e-f76235a3ec07 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:39.339716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1376/2760] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:13:39.347247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a93eb-0052-4167-8253-668d9e17b41a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:39.356663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95a93eb-0052-4167-8253-668d9e17b41a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:39.361504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cdff3f-d6ac-447d-8eb1-523f62dc8f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:39.362134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdff3f-d6ac-447d-8eb1-523f62dc8f1a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:39.362419 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdff3f-d6ac-447d-8eb1-523f62dc8f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:39.362692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdff3f-d6ac-447d-8eb1-523f62dc8f1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:39.363321 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cdff3f-d6ac-447d-8eb1-523f62dc8f1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:39.365267 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cdff3f-d6ac-447d-8eb1-523f62dc8f1a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:39.454906 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1385/2761] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:13:39 2022] POST /identity/v3/projects => generated 399 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:13:39.459526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb4580-805e-4499-801e-f76235a3ec07 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:39.466411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:39.472661 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cb4580-805e-4499-801e-f76235a3ec07 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:39.528066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f00559-74dc-4933-b263-957d8c42542d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:39.528685 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f00559-74dc-4933-b263-957d8c42542d admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:39.528966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f00559-74dc-4933-b263-957d8c42542d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:39.529243 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f00559-74dc-4933-b263-957d8c42542d admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:39.529874 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f00559-74dc-4933-b263-957d8c42542d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:39.532037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f00559-74dc-4933-b263-957d8c42542d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:39.632052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1377/2762] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:13:39 2022] POST /identity/v3/users => generated 300 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:13:39.635921 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cdff3f-d6ac-447d-8eb1-523f62dc8f1a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:39.647421 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cdff3f-d6ac-447d-8eb1-523f62dc8f1a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:39.652111 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc93ac-f8b0-4216-b94b-09d32f1d68e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:39.652711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc93ac-f8b0-4216-b94b-09d32f1d68e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:39.652993 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc93ac-f8b0-4216-b94b-09d32f1d68e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:39.653266 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bc93ac-f8b0-4216-b94b-09d32f1d68e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:39.653905 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc93ac-f8b0-4216-b94b-09d32f1d68e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:39.655494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bc93ac-f8b0-4216-b94b-09d32f1d68e0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:39.665824 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1386/2763] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:13:39.671038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f00559-74dc-4933-b263-957d8c42542d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:39.681754 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f00559-74dc-4933-b263-957d8c42542d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:39.687419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bc85ff-40d7-41ee-8b1c-a6498372be8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:39.689601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc85ff-40d7-41ee-8b1c-a6498372be8b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:39.689601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc85ff-40d7-41ee-8b1c-a6498372be8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:39.690486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bc85ff-40d7-41ee-8b1c-a6498372be8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:39.691147 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc85ff-40d7-41ee-8b1c-a6498372be8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:39.693139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bc85ff-40d7-41ee-8b1c-a6498372be8b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:39.705042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1378/2764] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:13:39.710249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc93ac-f8b0-4216-b94b-09d32f1d68e0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:39.720514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bc93ac-f8b0-4216-b94b-09d32f1d68e0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:39.725617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b40912-455f-4e6f-84b8-f680b09ae71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:39.726269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b40912-455f-4e6f-84b8-f680b09ae71d admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:39.726544 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b40912-455f-4e6f-84b8-f680b09ae71d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:39.726816 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b40912-455f-4e6f-84b8-f680b09ae71d admin admin] PATH_INFO: `/v3/projects/0066e11677214180b6d62f64d768c3a0/users/af45533511564dc991765977805bf7e0/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:39.742968 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b40912-455f-4e6f-84b8-f680b09ae71d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0066e11677214180b6d62f64d768c3a0, user_id=af45533511564dc991765977805bf7e0, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:39.745781 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b40912-455f-4e6f-84b8-f680b09ae71d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:39.843180 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1387/2765] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:13:39 2022] PUT /identity/v3/projects/0066e11677214180b6d62f64d768c3a0/users/af45533511564dc991765977805bf7e0/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:39.854182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b05f1-efc7-40ec-9a5d-73cf9304781f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:39.854490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b05f1-efc7-40ec-9a5d-73cf9304781f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:39.854785 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3b05f1-efc7-40ec-9a5d-73cf9304781f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:39.885191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7a3b05f1-efc7-40ec-9a5d-73cf9304781f None None] MFA Rules not processed for user `af45533511564dc991765977805bf7e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:39.886568 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7a3b05f1-efc7-40ec-9a5d-73cf9304781f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:39.967958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1379/2766] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:39 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:39.981929 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b40912-455f-4e6f-84b8-f680b09ae71d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:39.984708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c5b40912-455f-4e6f-84b8-f680b09ae71d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:40.055011 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b40912-455f-4e6f-84b8-f680b09ae71d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:40.059048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552011a-eff0-4186-b679-18bff38e9da7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:40.059690 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552011a-eff0-4186-b679-18bff38e9da7 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:40.060095 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552011a-eff0-4186-b679-18bff38e9da7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:40.060376 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f552011a-eff0-4186-b679-18bff38e9da7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:40.070154 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552011a-eff0-4186-b679-18bff38e9da7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:40.071861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f552011a-eff0-4186-b679-18bff38e9da7 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:40.082090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1388/2767] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:13:39 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:13:43.632406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3b985-224d-4952-aea2-f3f6a5a365b4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:43.632911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3b985-224d-4952-aea2-f3f6a5a365b4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:43.633111 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca3b985-224d-4952-aea2-f3f6a5a365b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:43.652571 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6ca3b985-224d-4952-aea2-f3f6a5a365b4 None None] MFA Rules not processed for user `4b1cdc37455c489fb45e1ce3e2421cc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:43.653780 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6ca3b985-224d-4952-aea2-f3f6a5a365b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:43.710178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1380/2768] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:43 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:50.076838 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=1,keystone:SELECT=55,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:51.522769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552011a-eff0-4186-b679-18bff38e9da7 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:51.589430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f552011a-eff0-4186-b679-18bff38e9da7 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:51.594121 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da7f56-8ceb-4720-b2cb-59ea87a9919d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:51.594807 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da7f56-8ceb-4720-b2cb-59ea87a9919d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:51.595090 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da7f56-8ceb-4720-b2cb-59ea87a9919d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:51.600250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80da7f56-8ceb-4720-b2cb-59ea87a9919d admin admin] PATH_INFO: `/v3/users/af45533511564dc991765977805bf7e0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:51.600250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da7f56-8ceb-4720-b2cb-59ea87a9919d admin admin] RBAC: Authorizing `identity:delete_user(user_id=af45533511564dc991765977805bf7e0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:51.600250 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80da7f56-8ceb-4720-b2cb-59ea87a9919d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:51.879480 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-80da7f56-8ceb-4720-b2cb-59ea87a9919d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af45533511564dc991765977805bf7e0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:51.990397 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-80da7f56-8ceb-4720-b2cb-59ea87a9919d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af45533511564dc991765977805bf7e0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:51.991051 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-80da7f56-8ceb-4720-b2cb-59ea87a9919d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af45533511564dc991765977805bf7e0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:51.991717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-80da7f56-8ceb-4720-b2cb-59ea87a9919d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af45533511564dc991765977805bf7e0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:52.005757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-80da7f56-8ceb-4720-b2cb-59ea87a9919d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af45533511564dc991765977805bf7e0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:52.018504 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1389/2769] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:13:51 2022] DELETE /identity/v3/users/af45533511564dc991765977805bf7e0 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:52.023192 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f5665-4b93-41f7-9759-749bc59b7ade None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:52.023713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f5665-4b93-41f7-9759-749bc59b7ade None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:52.023816 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f5665-4b93-41f7-9759-749bc59b7ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:52.044513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e71f5665-4b93-41f7-9759-749bc59b7ade None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:52.045653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e71f5665-4b93-41f7-9759-749bc59b7ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:52.082911 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:52.144057 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1381/2770] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:52.154896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da7f56-8ceb-4720-b2cb-59ea87a9919d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:52.156451 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-80da7f56-8ceb-4720-b2cb-59ea87a9919d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:52.221014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80da7f56-8ceb-4720-b2cb-59ea87a9919d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:52.224685 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952747dc-4bf7-4f7b-b477-627b70bb6aef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:52.225300 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952747dc-4bf7-4f7b-b477-627b70bb6aef service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:52.225582 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952747dc-4bf7-4f7b-b477-627b70bb6aef service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:52.225862 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952747dc-4bf7-4f7b-b477-627b70bb6aef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:52.235753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952747dc-4bf7-4f7b-b477-627b70bb6aef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:52.238055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952747dc-4bf7-4f7b-b477-627b70bb6aef service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:52.248711 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1390/2771] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:13:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:13:52.689028 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f5665-4b93-41f7-9759-749bc59b7ade None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:52.690647 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e71f5665-4b93-41f7-9759-749bc59b7ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:52.702475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f5665-4b93-41f7-9759-749bc59b7ade None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:52.707747 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e130c6-905b-4c8e-9335-64d31a0ab22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:52.708548 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e130c6-905b-4c8e-9335-64d31a0ab22a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:52.708836 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e130c6-905b-4c8e-9335-64d31a0ab22a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:52.709109 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e130c6-905b-4c8e-9335-64d31a0ab22a admin admin] PATH_INFO: `/v3/projects/0066e11677214180b6d62f64d768c3a0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:52.710290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e130c6-905b-4c8e-9335-64d31a0ab22a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0066e11677214180b6d62f64d768c3a0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:52.712264 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e130c6-905b-4c8e-9335-64d31a0ab22a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:52.814215 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d6e130c6-905b-4c8e-9335-64d31a0ab22a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0066e11677214180b6d62f64d768c3a0', 'request_id': 'req-d6e130c6-905b-4c8e-9335-64d31a0ab22a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:52.820688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d6e130c6-905b-4c8e-9335-64d31a0ab22a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0066e11677214180b6d62f64d768c3a0', 'request_id': 'req-d6e130c6-905b-4c8e-9335-64d31a0ab22a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:52.875922 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d6e130c6-905b-4c8e-9335-64d31a0ab22a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0066e11677214180b6d62f64d768c3a0', 'request_id': 'req-d6e130c6-905b-4c8e-9335-64d31a0ab22a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:52.887169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d6e130c6-905b-4c8e-9335-64d31a0ab22a admin admin] The token cache is being invalidate because project 0066e11677214180b6d62f64d768c3a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:13:52.887398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d6e130c6-905b-4c8e-9335-64d31a0ab22a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:52.888042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-d6e130c6-905b-4c8e-9335-64d31a0ab22a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:13:52.889547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1382/2772] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:13:52 2022] DELETE /identity/v3/projects/0066e11677214180b6d62f64d768c3a0 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:52.895209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736a882-e38b-4e91-a82c-ead0ad1cacf7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:52.895746 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736a882-e38b-4e91-a82c-ead0ad1cacf7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:52.895892 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3736a882-e38b-4e91-a82c-ead0ad1cacf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:52.915916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3736a882-e38b-4e91-a82c-ead0ad1cacf7 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:52.917119 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3736a882-e38b-4e91-a82c-ead0ad1cacf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:52.934064 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:52.979673 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1391/2773] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:53.043980 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e8309-2cfd-43ad-b37d-2f6845e8c07d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:53.044560 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e8309-2cfd-43ad-b37d-2f6845e8c07d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:53.044560 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6e8309-2cfd-43ad-b37d-2f6845e8c07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:53.066869 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2f6e8309-2cfd-43ad-b37d-2f6845e8c07d None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:53.068124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2f6e8309-2cfd-43ad-b37d-2f6845e8c07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:53.076097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1383/2774] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:13:53.080914 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736a882-e38b-4e91-a82c-ead0ad1cacf7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:53.090806 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3736a882-e38b-4e91-a82c-ead0ad1cacf7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:53.095474 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15ca41-f1ef-4e3f-b8a2-6cebaff09e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:53.096878 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15ca41-f1ef-4e3f-b8a2-6cebaff09e7d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:53.097155 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15ca41-f1ef-4e3f-b8a2-6cebaff09e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:53.097427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da15ca41-f1ef-4e3f-b8a2-6cebaff09e7d admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:53.098067 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15ca41-f1ef-4e3f-b8a2-6cebaff09e7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:13:53.098366 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15ca41-f1ef-4e3f-b8a2-6cebaff09e7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:53.099983 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da15ca41-f1ef-4e3f-b8a2-6cebaff09e7d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:53.115068 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1392/2775] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:13:53.116409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e8309-2cfd-43ad-b37d-2f6845e8c07d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:53.126411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6e8309-2cfd-43ad-b37d-2f6845e8c07d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:53.132634 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322f102-50a8-4ad6-8a5b-b8f6db7efc68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:53.133416 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322f102-50a8-4ad6-8a5b-b8f6db7efc68 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:53.133693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322f102-50a8-4ad6-8a5b-b8f6db7efc68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:53.133958 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322f102-50a8-4ad6-8a5b-b8f6db7efc68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:53.134574 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0322f102-50a8-4ad6-8a5b-b8f6db7efc68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:53.136655 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0322f102-50a8-4ad6-8a5b-b8f6db7efc68 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:53.266540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1384/2776] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:13:53 2022] POST /identity/v3/projects => generated 383 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:13:53.272179 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15ca41-f1ef-4e3f-b8a2-6cebaff09e7d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:53.282621 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da15ca41-f1ef-4e3f-b8a2-6cebaff09e7d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:53.340199 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075983dd-5bb4-4ef0-8eb4-c2451b61785e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:53.340808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075983dd-5bb4-4ef0-8eb4-c2451b61785e admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:53.341091 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075983dd-5bb4-4ef0-8eb4-c2451b61785e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:53.341365 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075983dd-5bb4-4ef0-8eb4-c2451b61785e admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:53.341998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075983dd-5bb4-4ef0-8eb4-c2451b61785e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:53.343938 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075983dd-5bb4-4ef0-8eb4-c2451b61785e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:53.419357 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1393/2777] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:13:53 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:13:53.423415 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322f102-50a8-4ad6-8a5b-b8f6db7efc68 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:53.434047 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322f102-50a8-4ad6-8a5b-b8f6db7efc68 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:53.438890 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a37e2-28d6-4116-b0b5-026604046cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:53.440156 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a37e2-28d6-4116-b0b5-026604046cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:53.440467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a37e2-28d6-4116-b0b5-026604046cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:53.440901 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a37e2-28d6-4116-b0b5-026604046cf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:53.441547 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a37e2-28d6-4116-b0b5-026604046cf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:53.443175 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a37e2-28d6-4116-b0b5-026604046cf7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:53.454750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1385/2778] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:13:53.468883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075983dd-5bb4-4ef0-8eb4-c2451b61785e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:53.480388 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075983dd-5bb4-4ef0-8eb4-c2451b61785e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:53.485418 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7194b5-36d5-475d-b188-4ce2366e355b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:53.486142 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7194b5-36d5-475d-b188-4ce2366e355b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:53.487033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7194b5-36d5-475d-b188-4ce2366e355b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:53.487443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7194b5-36d5-475d-b188-4ce2366e355b admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:53.488208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7194b5-36d5-475d-b188-4ce2366e355b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:53.490029 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7194b5-36d5-475d-b188-4ce2366e355b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:53.502841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1394/2779] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:13:53.506942 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a37e2-28d6-4116-b0b5-026604046cf7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:53.517124 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a37e2-28d6-4116-b0b5-026604046cf7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:53.522041 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453748c-cf9f-40f3-ab8c-5d6421126d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:53.522660 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453748c-cf9f-40f3-ab8c-5d6421126d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:53.522939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453748c-cf9f-40f3-ab8c-5d6421126d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:53.523214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453748c-cf9f-40f3-ab8c-5d6421126d84 admin admin] PATH_INFO: `/v3/projects/3b1fa05de673407eae9d3b50898c7b04/users/97774c7b322e49339d0006640db34e07/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:53.537998 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453748c-cf9f-40f3-ab8c-5d6421126d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b1fa05de673407eae9d3b50898c7b04, user_id=97774c7b322e49339d0006640db34e07, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:53.540713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453748c-cf9f-40f3-ab8c-5d6421126d84 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:53.602293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1386/2780] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:13:53 2022] PUT /identity/v3/projects/3b1fa05de673407eae9d3b50898c7b04/users/97774c7b322e49339d0006640db34e07/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:13:53.607453 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211bee5-d7dd-4b81-a78e-ed9706c32855 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:53.607767 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211bee5-d7dd-4b81-a78e-ed9706c32855 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:53.608041 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211bee5-d7dd-4b81-a78e-ed9706c32855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:53.628655 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b211bee5-d7dd-4b81-a78e-ed9706c32855 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:53.629860 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b211bee5-d7dd-4b81-a78e-ed9706c32855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:53.692737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1395/2781] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:13:53.702718 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453748c-cf9f-40f3-ab8c-5d6421126d84 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:53.704562 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f453748c-cf9f-40f3-ab8c-5d6421126d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:53.747402 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:53.765989 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453748c-cf9f-40f3-ab8c-5d6421126d84 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:53.771030 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f625495-cbb6-44fb-99de-40de75f6d11f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:53.771662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f625495-cbb6-44fb-99de-40de75f6d11f service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:53.771971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f625495-cbb6-44fb-99de-40de75f6d11f service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:53.772225 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f625495-cbb6-44fb-99de-40de75f6d11f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:53.782341 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f625495-cbb6-44fb-99de-40de75f6d11f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:53.783848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f625495-cbb6-44fb-99de-40de75f6d11f service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:53.793942 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1387/2782] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:13:59.493703 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a72afc-e71e-4812-bfa3-072e2184b905 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:59.494238 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a72afc-e71e-4812-bfa3-072e2184b905 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:59.497543 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a72afc-e71e-4812-bfa3-072e2184b905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:59.531546 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a4a72afc-e71e-4812-bfa3-072e2184b905 None None] MFA Rules not processed for user `97774c7b322e49339d0006640db34e07`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:13:59.532821 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a4a72afc-e71e-4812-bfa3-072e2184b905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:13:59.553993 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:13:59.623218 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1396/2783] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:13:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:13:59.628589 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f625495-cbb6-44fb-99de-40de75f6d11f service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:59.638513 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f625495-cbb6-44fb-99de-40de75f6d11f service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:59.645078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc7c5f-5cc1-4c02-a333-944b63927db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:59.645078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc7c5f-5cc1-4c02-a333-944b63927db7 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:59.645078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc7c5f-5cc1-4c02-a333-944b63927db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:59.645078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc7c5f-5cc1-4c02-a333-944b63927db7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:59.645635 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc7c5f-5cc1-4c02-a333-944b63927db7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:59.647523 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc7c5f-5cc1-4c02-a333-944b63927db7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:59.736673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1388/2784] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:13:59 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:13:59.740534 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a72afc-e71e-4812-bfa3-072e2184b905 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:59.757192 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a72afc-e71e-4812-bfa3-072e2184b905 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:59.824321 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e09e4-3a49-4a34-8541-7eef76490686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:59.824942 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e09e4-3a49-4a34-8541-7eef76490686 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:59.825378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e09e4-3a49-4a34-8541-7eef76490686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:59.825660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32e09e4-3a49-4a34-8541-7eef76490686 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:59.826314 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e09e4-3a49-4a34-8541-7eef76490686 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:59.828329 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32e09e4-3a49-4a34-8541-7eef76490686 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:13:59.935707 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1397/2785] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:13:59 2022] POST /identity/v3/users => generated 298 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:13:59.939688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc7c5f-5cc1-4c02-a333-944b63927db7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:13:59.952308 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc7c5f-5cc1-4c02-a333-944b63927db7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:13:59.959837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b6185-9571-4832-9441-3a35c4c3cc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:13:59.960459 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b6185-9571-4832-9441-3a35c4c3cc12 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:13:59.960734 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b6185-9571-4832-9441-3a35c4c3cc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:13:59.961004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b6185-9571-4832-9441-3a35c4c3cc12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:13:59.961705 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b6185-9571-4832-9441-3a35c4c3cc12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:13:59.964399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574b6185-9571-4832-9441-3a35c4c3cc12 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:14:00.011314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1389/2786] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:13:59 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:14:00.014528 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e09e4-3a49-4a34-8541-7eef76490686 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:14:00.030478 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32e09e4-3a49-4a34-8541-7eef76490686 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:14:00.045442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807795a-9890-4fa7-b565-463e20bfeee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:14:00.046338 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807795a-9890-4fa7-b565-463e20bfeee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:00.047408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807795a-9890-4fa7-b565-463e20bfeee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:00.047408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d807795a-9890-4fa7-b565-463e20bfeee9 admin admin] PATH_INFO: `/v3/projects/de31ca54ad8e46d29d9a868ce8c54afe/users/c26340f795144d9a8f5d8204bbb1114c/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:00.076315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807795a-9890-4fa7-b565-463e20bfeee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de31ca54ad8e46d29d9a868ce8c54afe, user_id=c26340f795144d9a8f5d8204bbb1114c, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:14:00.079264 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d807795a-9890-4fa7-b565-463e20bfeee9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:14:00.221700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1398/2787] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:14:00 2022] PUT /identity/v3/projects/de31ca54ad8e46d29d9a868ce8c54afe/users/c26340f795144d9a8f5d8204bbb1114c/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:14:00.225821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b6185-9571-4832-9441-3a35c4c3cc12 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:14:00.237933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b6185-9571-4832-9441-3a35c4c3cc12 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:14:00.310217 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e8cbb6-8b0c-4615-9b26-d0b10bf97281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:14:00.310839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8cbb6-8b0c-4615-9b26-d0b10bf97281 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:00.311120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8cbb6-8b0c-4615-9b26-d0b10bf97281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:00.311394 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e8cbb6-8b0c-4615-9b26-d0b10bf97281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:00.312143 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8cbb6-8b0c-4615-9b26-d0b10bf97281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:14:00.313779 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e8cbb6-8b0c-4615-9b26-d0b10bf97281 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:14:00.324896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1390/2788] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:14:00.329285 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807795a-9890-4fa7-b565-463e20bfeee9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:14:00.342121 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d807795a-9890-4fa7-b565-463e20bfeee9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:14:00.351616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc653ff1-bdc1-4137-b3f7-7685cf996ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:14:00.351616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc653ff1-bdc1-4137-b3f7-7685cf996ee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:00.351616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc653ff1-bdc1-4137-b3f7-7685cf996ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:00.351616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc653ff1-bdc1-4137-b3f7-7685cf996ee3 admin admin] PATH_INFO: `/v3/projects/de31ca54ad8e46d29d9a868ce8c54afe/users/c26340f795144d9a8f5d8204bbb1114c/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:00.352582 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc653ff1-bdc1-4137-b3f7-7685cf996ee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de31ca54ad8e46d29d9a868ce8c54afe, user_id=c26340f795144d9a8f5d8204bbb1114c, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:14:00.355430 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc653ff1-bdc1-4137-b3f7-7685cf996ee3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:14:00.368932 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1399/2789] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:14:00 2022] PUT /identity/v3/projects/de31ca54ad8e46d29d9a868ce8c54afe/users/c26340f795144d9a8f5d8204bbb1114c/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:14:08.570122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de225e-17fa-4b46-8492-b42a0c6112b2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:08.570687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de225e-17fa-4b46-8492-b42a0c6112b2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:08.570687 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6de225e-17fa-4b46-8492-b42a0c6112b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:08.603260 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b6de225e-17fa-4b46-8492-b42a0c6112b2 None None] MFA Rules not processed for user `c26340f795144d9a8f5d8204bbb1114c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:14:08.605072 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b6de225e-17fa-4b46-8492-b42a0c6112b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:14:08.687088 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1391/2790] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:14:08 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:14:08.699899 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc653ff1-bdc1-4137-b3f7-7685cf996ee3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:14:08.701687 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cc653ff1-bdc1-4137-b3f7-7685cf996ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:14:08.761956 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc653ff1-bdc1-4137-b3f7-7685cf996ee3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:14:08.765573 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ee2a9-246e-494b-93c4-7f6423813297 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6dee005fc3d4fb593a1771e7851c20a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:14:08.766189 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ee2a9-246e-494b-93c4-7f6423813297 service glance] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:08.766472 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ee2a9-246e-494b-93c4-7f6423813297 service glance] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:08.766748 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ee2a9-246e-494b-93c4-7f6423813297 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:08.795188 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:14:08.827442 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ee2a9-246e-494b-93c4-7f6423813297 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:14:08.835824 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ee2a9-246e-494b-93c4-7f6423813297 service glance] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:14:08.839007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1400/2791] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:14:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:14:09.226619 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c780d-0033-4368-a8d6-ff01aef25caa None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:09.227529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c780d-0033-4368-a8d6-ff01aef25caa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:09.227529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c780d-0033-4368-a8d6-ff01aef25caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:09.276265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:14:09.285960 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-019c780d-0033-4368-a8d6-ff01aef25caa None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:14:09.287194 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-019c780d-0033-4368-a8d6-ff01aef25caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:14:09.383179 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1392/2792] 10.222.0.67 () {58 vars in 1009 bytes} [Mon Sep 26 18:14:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:14:09.545385 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ee2a9-246e-494b-93c4-7f6423813297 service glance] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:14:09.546854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f49ee2a9-246e-494b-93c4-7f6423813297 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:14:09.649248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ee2a9-246e-494b-93c4-7f6423813297 service glance] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:14:09.655178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b704c0d-c390-451e-bc5d-fb8227101f63 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:14:09.655854 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b704c0d-c390-451e-bc5d-fb8227101f63 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:09.656136 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b704c0d-c390-451e-bc5d-fb8227101f63 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:09.656409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b704c0d-c390-451e-bc5d-fb8227101f63 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:09.666446 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b704c0d-c390-451e-bc5d-fb8227101f63 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:14:09.668105 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b704c0d-c390-451e-bc5d-fb8227101f63 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:14:09.676348 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1401/2793] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:14:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:14:19.607450 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=28 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:14:19.672966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=57 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:14:20.761028 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7bc85-5fb0-4bdf-8489-e761cdd83dac None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:20.761677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7bc85-5fb0-4bdf-8489-e761cdd83dac None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:20.761677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7bc85-5fb0-4bdf-8489-e761cdd83dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:20.780896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-40e7bc85-5fb0-4bdf-8489-e761cdd83dac None None] MFA Rules not processed for user `97774c7b322e49339d0006640db34e07`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:14:20.782111 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-40e7bc85-5fb0-4bdf-8489-e761cdd83dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:14:20.789020 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1393/2794] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:14:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:14:24.413183 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b704c0d-c390-451e-bc5d-fb8227101f63 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:14:24.472804 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b704c0d-c390-451e-bc5d-fb8227101f63 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:14:24.477350 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c08be4-6e66-409b-b7ac-4cf8075cfd49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:14:24.477658 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c08be4-6e66-409b-b7ac-4cf8075cfd49 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:24.477880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c08be4-6e66-409b-b7ac-4cf8075cfd49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:24.478203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c08be4-6e66-409b-b7ac-4cf8075cfd49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:24.480242 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-75c08be4-6e66-409b-b7ac-4cf8075cfd49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:14:24.491488 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c08be4-6e66-409b-b7ac-4cf8075cfd49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:14:24.493771 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c08be4-6e66-409b-b7ac-4cf8075cfd49 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:14:24.504764 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1402/2795] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:14:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:14:26.508031 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c0ad8-ec76-4eda-8631-debc6989f839 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:26.508621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c0ad8-ec76-4eda-8631-debc6989f839 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:26.508746 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c0ad8-ec76-4eda-8631-debc6989f839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:26.530047 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d13c0ad8-ec76-4eda-8631-debc6989f839 None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:14:26.531256 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d13c0ad8-ec76-4eda-8631-debc6989f839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:14:26.538962 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1394/2796] 10.222.0.67 () {58 vars in 1009 bytes} [Mon Sep 26 18:14:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:14:26.552536 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c08be4-6e66-409b-b7ac-4cf8075cfd49 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:14:26.566014 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c08be4-6e66-409b-b7ac-4cf8075cfd49 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:14:26.571104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089fa10-15b0-4589-aec3-166b83b775da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:14:26.571743 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089fa10-15b0-4589-aec3-166b83b775da service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:26.572052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089fa10-15b0-4589-aec3-166b83b775da service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:26.572321 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1089fa10-15b0-4589-aec3-166b83b775da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:26.581904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1089fa10-15b0-4589-aec3-166b83b775da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:14:26.583765 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1089fa10-15b0-4589-aec3-166b83b775da service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:14:26.592614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1403/2797] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:14:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 26 18:14:36.136985 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c0ad8-ec76-4eda-8631-debc6989f839 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:14:36.139838 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d13c0ad8-ec76-4eda-8631-debc6989f839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:14:36.213938 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c0ad8-ec76-4eda-8631-debc6989f839 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:14:36.219443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0903aa8-f845-440e-a938-13f5ab6f8177 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:14:36.220092 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0903aa8-f845-440e-a938-13f5ab6f8177 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:36.220374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0903aa8-f845-440e-a938-13f5ab6f8177 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:36.220749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0903aa8-f845-440e-a938-13f5ab6f8177 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:36.222763 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b0903aa8-f845-440e-a938-13f5ab6f8177 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:14:36.235021 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0903aa8-f845-440e-a938-13f5ab6f8177 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:14:36.237458 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0903aa8-f845-440e-a938-13f5ab6f8177 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:14:36.248239 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1395/2798] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:14:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:14:36.588916 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=32 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:14:38.445134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089fa10-15b0-4589-aec3-166b83b775da service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:14:38.446657 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1089fa10-15b0-4589-aec3-166b83b775da service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:14:38.517931 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1089fa10-15b0-4589-aec3-166b83b775da service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:14:38.523053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2771e-8105-4ef0-8f91-8bbe9299ae5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:14:38.523710 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2771e-8105-4ef0-8f91-8bbe9299ae5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:38.523992 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2771e-8105-4ef0-8f91-8bbe9299ae5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:38.524261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e2771e-8105-4ef0-8f91-8bbe9299ae5d admin admin] PATH_INFO: `/v3/users/ecf12b3f54e349d5a8f6587410c79501` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:38.525485 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2771e-8105-4ef0-8f91-8bbe9299ae5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf12b3f54e349d5a8f6587410c79501)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:14:38.527467 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e2771e-8105-4ef0-8f91-8bbe9299ae5d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:14:38.702982 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-33e2771e-8105-4ef0-8f91-8bbe9299ae5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf12b3f54e349d5a8f6587410c79501'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:14:38.778834 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-33e2771e-8105-4ef0-8f91-8bbe9299ae5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf12b3f54e349d5a8f6587410c79501'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:14:38.779464 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-33e2771e-8105-4ef0-8f91-8bbe9299ae5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf12b3f54e349d5a8f6587410c79501'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:14:38.780111 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-33e2771e-8105-4ef0-8f91-8bbe9299ae5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf12b3f54e349d5a8f6587410c79501'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:14:38.793053 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-33e2771e-8105-4ef0-8f91-8bbe9299ae5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf12b3f54e349d5a8f6587410c79501'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:14:38.805377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1404/2799] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:14:38 2022] DELETE /identity/v3/users/ecf12b3f54e349d5a8f6587410c79501 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:14:39.077811 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0903aa8-f845-440e-a938-13f5ab6f8177 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:14:39.088757 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0903aa8-f845-440e-a938-13f5ab6f8177 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:14:39.149004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb0575-5cf5-422d-af9a-da112c9afe6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:14:39.149637 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb0575-5cf5-422d-af9a-da112c9afe6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:39.149919 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb0575-5cf5-422d-af9a-da112c9afe6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:39.150191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbb0575-5cf5-422d-af9a-da112c9afe6e admin admin] PATH_INFO: `/v3/projects/518090209d9a462fbf40d6cb86c3c695` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:39.151319 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb0575-5cf5-422d-af9a-da112c9afe6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=518090209d9a462fbf40d6cb86c3c695)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:14:39.153240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbb0575-5cf5-422d-af9a-da112c9afe6e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:14:39.271330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0fbb0575-5cf5-422d-af9a-da112c9afe6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '518090209d9a462fbf40d6cb86c3c695', 'request_id': 'req-0fbb0575-5cf5-422d-af9a-da112c9afe6e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:14:39.277678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0fbb0575-5cf5-422d-af9a-da112c9afe6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '518090209d9a462fbf40d6cb86c3c695', 'request_id': 'req-0fbb0575-5cf5-422d-af9a-da112c9afe6e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:14:39.350009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0fbb0575-5cf5-422d-af9a-da112c9afe6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '518090209d9a462fbf40d6cb86c3c695', 'request_id': 'req-0fbb0575-5cf5-422d-af9a-da112c9afe6e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:14:39.360999 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0fbb0575-5cf5-422d-af9a-da112c9afe6e admin admin] The token cache is being invalidate because project 518090209d9a462fbf40d6cb86c3c695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:14:39.362146 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0fbb0575-5cf5-422d-af9a-da112c9afe6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:14:39.363067 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0fbb0575-5cf5-422d-af9a-da112c9afe6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:14:39.364507 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1396/2800] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:14:39 2022] DELETE /identity/v3/projects/518090209d9a462fbf40d6cb86c3c695 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:14:43.171932 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2771e-8105-4ef0-8f91-8bbe9299ae5d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:14:43.173468 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-33e2771e-8105-4ef0-8f91-8bbe9299ae5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:14:43.248526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e2771e-8105-4ef0-8f91-8bbe9299ae5d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:14:43.261859 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:14:43.315370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743721ee-4b53-4cbb-bcae-eb49c7341584 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:14:43.316377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743721ee-4b53-4cbb-bcae-eb49c7341584 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:43.316722 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743721ee-4b53-4cbb-bcae-eb49c7341584 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:43.317592 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743721ee-4b53-4cbb-bcae-eb49c7341584 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:43.319097 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-743721ee-4b53-4cbb-bcae-eb49c7341584 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:14:43.333429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743721ee-4b53-4cbb-bcae-eb49c7341584 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:14:43.335166 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743721ee-4b53-4cbb-bcae-eb49c7341584 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:14:43.346211 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1405/2801] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:14:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:14:47.421572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb0575-5cf5-422d-af9a-da112c9afe6e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:14:47.422964 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0fbb0575-5cf5-422d-af9a-da112c9afe6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:14:47.429225 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:14:47.483061 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbb0575-5cf5-422d-af9a-da112c9afe6e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:14:47.487619 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9092dc-6caf-43f1-9e14-b2264ed9b794 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:14:47.488273 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9092dc-6caf-43f1-9e14-b2264ed9b794 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:14:47.488555 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9092dc-6caf-43f1-9e14-b2264ed9b794 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:14:47.488829 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9092dc-6caf-43f1-9e14-b2264ed9b794 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:14:47.490543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4c9092dc-6caf-43f1-9e14-b2264ed9b794 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:14:47.550422 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9092dc-6caf-43f1-9e14-b2264ed9b794 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:14:47.552231 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9092dc-6caf-43f1-9e14-b2264ed9b794 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:14:47.561374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1397/2802] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:14:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:14:53.452691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=26 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:14:57.590966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=47 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:01.915769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743721ee-4b53-4cbb-bcae-eb49c7341584 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:01.917346 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-743721ee-4b53-4cbb-bcae-eb49c7341584 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:01.982214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743721ee-4b53-4cbb-bcae-eb49c7341584 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:01.986936 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b5388-7e69-4c6d-8569-d6121bab67e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:01.987730 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b5388-7e69-4c6d-8569-d6121bab67e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:01.988012 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b5388-7e69-4c6d-8569-d6121bab67e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:01.988279 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785b5388-7e69-4c6d-8569-d6121bab67e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:01.990234 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-785b5388-7e69-4c6d-8569-d6121bab67e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:02.056247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b5388-7e69-4c6d-8569-d6121bab67e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:02.059922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785b5388-7e69-4c6d-8569-d6121bab67e5 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:02.067611 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1406/2803] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:15:01 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:15:12.063345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=50 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:22.913216 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9092dc-6caf-43f1-9e14-b2264ed9b794 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:22.914655 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4c9092dc-6caf-43f1-9e14-b2264ed9b794 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:22.927279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9092dc-6caf-43f1-9e14-b2264ed9b794 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:22.930990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca9d7f-7e45-4ded-8d44-1ae6d5995980 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:22.931674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca9d7f-7e45-4ded-8d44-1ae6d5995980 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:22.931973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca9d7f-7e45-4ded-8d44-1ae6d5995980 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:22.932250 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca9d7f-7e45-4ded-8d44-1ae6d5995980 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:22.934016 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ffca9d7f-7e45-4ded-8d44-1ae6d5995980 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:22.999181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca9d7f-7e45-4ded-8d44-1ae6d5995980 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:23.000658 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca9d7f-7e45-4ded-8d44-1ae6d5995980 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:23.011116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1398/2804] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:15:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:15:26.898714 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b5388-7e69-4c6d-8569-d6121bab67e5 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:26.901002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-785b5388-7e69-4c6d-8569-d6121bab67e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:26.914576 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785b5388-7e69-4c6d-8569-d6121bab67e5 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:26.919393 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e70ec5-0beb-4f8f-90b3-6500e0803ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:26.920126 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e70ec5-0beb-4f8f-90b3-6500e0803ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:26.920368 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e70ec5-0beb-4f8f-90b3-6500e0803ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:26.920644 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e70ec5-0beb-4f8f-90b3-6500e0803ab5 admin admin] PATH_INFO: `/v3/users/4b1cdc37455c489fb45e1ce3e2421cc0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:26.921769 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e70ec5-0beb-4f8f-90b3-6500e0803ab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b1cdc37455c489fb45e1ce3e2421cc0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:26.923694 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e70ec5-0beb-4f8f-90b3-6500e0803ab5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:27.134555 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c6e70ec5-0beb-4f8f-90b3-6500e0803ab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b1cdc37455c489fb45e1ce3e2421cc0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:27.230203 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c6e70ec5-0beb-4f8f-90b3-6500e0803ab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b1cdc37455c489fb45e1ce3e2421cc0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:27.230843 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c6e70ec5-0beb-4f8f-90b3-6500e0803ab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b1cdc37455c489fb45e1ce3e2421cc0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:27.231464 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c6e70ec5-0beb-4f8f-90b3-6500e0803ab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b1cdc37455c489fb45e1ce3e2421cc0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:27.244741 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c6e70ec5-0beb-4f8f-90b3-6500e0803ab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b1cdc37455c489fb45e1ce3e2421cc0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:27.257514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1407/2805] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:15:26 2022] DELETE /identity/v3/users/4b1cdc37455c489fb45e1ce3e2421cc0 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:15:27.518398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca9d7f-7e45-4ded-8d44-1ae6d5995980 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:27.529163 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca9d7f-7e45-4ded-8d44-1ae6d5995980 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:27.588374 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da61352a-b6dd-496e-bae3-8c64a46e6db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:27.588970 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61352a-b6dd-496e-bae3-8c64a46e6db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:27.589251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61352a-b6dd-496e-bae3-8c64a46e6db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:27.594419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da61352a-b6dd-496e-bae3-8c64a46e6db3 admin admin] PATH_INFO: `/v3/projects/7750acf9ab4843d1a3848472e99881a1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:27.594419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da61352a-b6dd-496e-bae3-8c64a46e6db3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7750acf9ab4843d1a3848472e99881a1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:27.594419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da61352a-b6dd-496e-bae3-8c64a46e6db3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:27.747686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-da61352a-b6dd-496e-bae3-8c64a46e6db3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7750acf9ab4843d1a3848472e99881a1', 'request_id': 'req-da61352a-b6dd-496e-bae3-8c64a46e6db3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:27.754236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-da61352a-b6dd-496e-bae3-8c64a46e6db3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7750acf9ab4843d1a3848472e99881a1', 'request_id': 'req-da61352a-b6dd-496e-bae3-8c64a46e6db3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:27.835343 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-da61352a-b6dd-496e-bae3-8c64a46e6db3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7750acf9ab4843d1a3848472e99881a1', 'request_id': 'req-da61352a-b6dd-496e-bae3-8c64a46e6db3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:27.847786 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-da61352a-b6dd-496e-bae3-8c64a46e6db3 admin admin] The token cache is being invalidate because project 7750acf9ab4843d1a3848472e99881a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:15:27.848108 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-da61352a-b6dd-496e-bae3-8c64a46e6db3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:27.848760 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-da61352a-b6dd-496e-bae3-8c64a46e6db3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:27.850609 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1399/2806] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:15:27 2022] DELETE /identity/v3/projects/7750acf9ab4843d1a3848472e99881a1 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:15:27.855815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3df53-8b5b-4d50-901f-643a0cba535b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:27.856106 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3df53-8b5b-4d50-901f-643a0cba535b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:27.856410 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a3df53-8b5b-4d50-901f-643a0cba535b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:27.877223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-87a3df53-8b5b-4d50-901f-643a0cba535b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:15:27.878462 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-87a3df53-8b5b-4d50-901f-643a0cba535b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:27.938521 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1408/2807] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:15:27.946464 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf214bd-8cde-450f-a03b-d9844888d6e4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:27.946852 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf214bd-8cde-450f-a03b-d9844888d6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:27.947710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf214bd-8cde-450f-a03b-d9844888d6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:27.967488 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4bf214bd-8cde-450f-a03b-d9844888d6e4 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:15:27.968727 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4bf214bd-8cde-450f-a03b-d9844888d6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:27.977759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1400/2808] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:15:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:15:27.981898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3df53-8b5b-4d50-901f-643a0cba535b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:27.991550 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a3df53-8b5b-4d50-901f-643a0cba535b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:27.997301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f5354-4b4c-460c-8085-51a78c815441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:27.997459 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f5354-4b4c-460c-8085-51a78c815441 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:27.997695 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f5354-4b4c-460c-8085-51a78c815441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:27.997972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557f5354-4b4c-460c-8085-51a78c815441 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:27.998614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f5354-4b4c-460c-8085-51a78c815441 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:15:27.998919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f5354-4b4c-460c-8085-51a78c815441 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:28.000586 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557f5354-4b4c-460c-8085-51a78c815441 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:28.013134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1409/2809] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:15:28.017481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf214bd-8cde-450f-a03b-d9844888d6e4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:28.028784 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf214bd-8cde-450f-a03b-d9844888d6e4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:28.033654 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2aada-2f00-413d-b465-73eacf91a270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:28.034395 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2aada-2f00-413d-b465-73eacf91a270 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:28.034678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2aada-2f00-413d-b465-73eacf91a270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:28.034951 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d2aada-2f00-413d-b465-73eacf91a270 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:28.035592 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d2aada-2f00-413d-b465-73eacf91a270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:28.037570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d2aada-2f00-413d-b465-73eacf91a270 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:28.053788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:28.128104 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1401/2810] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:15:28 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:15:28.128767 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f5354-4b4c-460c-8085-51a78c815441 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:28.140338 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557f5354-4b4c-460c-8085-51a78c815441 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:28.198618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669dc77-fe1b-4ffb-86e1-b7f5fedaa1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:28.199235 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669dc77-fe1b-4ffb-86e1-b7f5fedaa1da admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:28.199517 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669dc77-fe1b-4ffb-86e1-b7f5fedaa1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:28.199819 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669dc77-fe1b-4ffb-86e1-b7f5fedaa1da admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:28.200445 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669dc77-fe1b-4ffb-86e1-b7f5fedaa1da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:28.202345 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1669dc77-fe1b-4ffb-86e1-b7f5fedaa1da admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:28.210496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:28.287007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1410/2811] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:15:28 2022] POST /identity/v3/users => generated 293 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:15:28.291836 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2aada-2f00-413d-b465-73eacf91a270 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:28.302209 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d2aada-2f00-413d-b465-73eacf91a270 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:28.307589 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d87bcf-a4f7-4cc8-a55b-1428bf0488c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:28.308240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d87bcf-a4f7-4cc8-a55b-1428bf0488c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:28.308504 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d87bcf-a4f7-4cc8-a55b-1428bf0488c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:28.308805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d87bcf-a4f7-4cc8-a55b-1428bf0488c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:28.309440 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d87bcf-a4f7-4cc8-a55b-1428bf0488c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:28.311037 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d87bcf-a4f7-4cc8-a55b-1428bf0488c0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:28.321167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1402/2812] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:15:28.325473 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669dc77-fe1b-4ffb-86e1-b7f5fedaa1da admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:28.335853 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1669dc77-fe1b-4ffb-86e1-b7f5fedaa1da admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:28.340653 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4941618-5895-4dbc-9592-49cfa8af03e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:28.341257 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4941618-5895-4dbc-9592-49cfa8af03e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:28.341540 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4941618-5895-4dbc-9592-49cfa8af03e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:28.341815 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4941618-5895-4dbc-9592-49cfa8af03e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:28.342461 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4941618-5895-4dbc-9592-49cfa8af03e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:28.344135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4941618-5895-4dbc-9592-49cfa8af03e0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:28.356282 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1411/2813] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:15:28.360477 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d87bcf-a4f7-4cc8-a55b-1428bf0488c0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:28.371055 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d87bcf-a4f7-4cc8-a55b-1428bf0488c0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:28.375742 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bab74c-832a-4527-a8b0-d7e44ecc96db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:28.376398 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bab74c-832a-4527-a8b0-d7e44ecc96db admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:28.376677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bab74c-832a-4527-a8b0-d7e44ecc96db admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:28.376953 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bab74c-832a-4527-a8b0-d7e44ecc96db admin admin] PATH_INFO: `/v3/projects/909137c2da3f40b4bbb1072a22965ada/users/ea5d62d563924fcd89427ad0a0166c1b/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:28.392080 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bab74c-832a-4527-a8b0-d7e44ecc96db admin admin] RBAC: Authorizing `identity:create_grant(project_id=909137c2da3f40b4bbb1072a22965ada, user_id=ea5d62d563924fcd89427ad0a0166c1b, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:28.394716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bab74c-832a-4527-a8b0-d7e44ecc96db admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:28.440810 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1403/2814] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:15:28 2022] PUT /identity/v3/projects/909137c2da3f40b4bbb1072a22965ada/users/ea5d62d563924fcd89427ad0a0166c1b/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:15:28.451422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8caeb-945f-4f44-8331-5d5d4432ab7a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:28.451838 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8caeb-945f-4f44-8331-5d5d4432ab7a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:28.452120 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8caeb-945f-4f44-8331-5d5d4432ab7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:28.480598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b6c8caeb-945f-4f44-8331-5d5d4432ab7a None None] MFA Rules not processed for user `ea5d62d563924fcd89427ad0a0166c1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:15:28.481803 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b6c8caeb-945f-4f44-8331-5d5d4432ab7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:28.547232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1412/2815] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:15:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:15:28.558961 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bab74c-832a-4527-a8b0-d7e44ecc96db admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:28.560704 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-98bab74c-832a-4527-a8b0-d7e44ecc96db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:28.633786 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bab74c-832a-4527-a8b0-d7e44ecc96db admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:28.639540 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67895bb2-27ad-47a3-8384-fcaf5c24d1de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:28.640172 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67895bb2-27ad-47a3-8384-fcaf5c24d1de service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:28.640454 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67895bb2-27ad-47a3-8384-fcaf5c24d1de service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:28.640726 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67895bb2-27ad-47a3-8384-fcaf5c24d1de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:28.648879 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67895bb2-27ad-47a3-8384-fcaf5c24d1de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:28.651163 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67895bb2-27ad-47a3-8384-fcaf5c24d1de service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:28.659927 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1404/2816] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:15:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:15:37.812768 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8caeb-945f-4f44-8331-5d5d4432ab7a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:37.875007 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8caeb-945f-4f44-8331-5d5d4432ab7a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:37.881003 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d58d2-0a71-4f30-b698-b26dbc1fdc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:37.881631 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d58d2-0a71-4f30-b698-b26dbc1fdc54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:37.881923 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d58d2-0a71-4f30-b698-b26dbc1fdc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:37.882189 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d58d2-0a71-4f30-b698-b26dbc1fdc54 admin admin] PATH_INFO: `/v3/users/ea5d62d563924fcd89427ad0a0166c1b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:37.883304 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d58d2-0a71-4f30-b698-b26dbc1fdc54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea5d62d563924fcd89427ad0a0166c1b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:37.885235 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d58d2-0a71-4f30-b698-b26dbc1fdc54 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:38.106737 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-661d58d2-0a71-4f30-b698-b26dbc1fdc54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea5d62d563924fcd89427ad0a0166c1b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:38.172027 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-661d58d2-0a71-4f30-b698-b26dbc1fdc54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea5d62d563924fcd89427ad0a0166c1b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:38.172646 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-661d58d2-0a71-4f30-b698-b26dbc1fdc54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea5d62d563924fcd89427ad0a0166c1b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:38.173280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-661d58d2-0a71-4f30-b698-b26dbc1fdc54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea5d62d563924fcd89427ad0a0166c1b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:38.184946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:38.186277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-661d58d2-0a71-4f30-b698-b26dbc1fdc54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea5d62d563924fcd89427ad0a0166c1b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:38.199317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1413/2817] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:15:37 2022] DELETE /identity/v3/users/ea5d62d563924fcd89427ad0a0166c1b => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:15:38.203974 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af325f19-c9f2-4c98-8dc7-d0f9a1805a7c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:38.204475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af325f19-c9f2-4c98-8dc7-d0f9a1805a7c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:38.204475 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af325f19-c9f2-4c98-8dc7-d0f9a1805a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:38.224467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-af325f19-c9f2-4c98-8dc7-d0f9a1805a7c None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:15:38.226942 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-af325f19-c9f2-4c98-8dc7-d0f9a1805a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:38.286068 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1405/2818] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:15:38.296141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d58d2-0a71-4f30-b698-b26dbc1fdc54 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:38.297514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-661d58d2-0a71-4f30-b698-b26dbc1fdc54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:38.355225 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d58d2-0a71-4f30-b698-b26dbc1fdc54 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:38.358736 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa37083-c8bc-48c1-927f-1d547cfb36ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:38.359373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa37083-c8bc-48c1-927f-1d547cfb36ac service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:38.359668 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa37083-c8bc-48c1-927f-1d547cfb36ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:38.359964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa37083-c8bc-48c1-927f-1d547cfb36ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:38.369027 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa37083-c8bc-48c1-927f-1d547cfb36ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:38.371113 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa37083-c8bc-48c1-927f-1d547cfb36ac service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:38.380708 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1414/2819] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:15:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:15:38.956629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af325f19-c9f2-4c98-8dc7-d0f9a1805a7c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:38.958104 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-af325f19-c9f2-4c98-8dc7-d0f9a1805a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:38.970622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af325f19-c9f2-4c98-8dc7-d0f9a1805a7c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:38.975971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb50b12-9dbd-4a1f-9697-e12cba91600b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:38.975971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb50b12-9dbd-4a1f-9697-e12cba91600b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:38.976334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb50b12-9dbd-4a1f-9697-e12cba91600b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:38.976621 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb50b12-9dbd-4a1f-9697-e12cba91600b admin admin] PATH_INFO: `/v3/projects/909137c2da3f40b4bbb1072a22965ada` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:38.981567 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb50b12-9dbd-4a1f-9697-e12cba91600b admin admin] RBAC: Authorizing `identity:delete_project(project_id=909137c2da3f40b4bbb1072a22965ada)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:38.983552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb50b12-9dbd-4a1f-9697-e12cba91600b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:39.139600 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:39.145646 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cdb50b12-9dbd-4a1f-9697-e12cba91600b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '909137c2da3f40b4bbb1072a22965ada', 'request_id': 'req-cdb50b12-9dbd-4a1f-9697-e12cba91600b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:39.152236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cdb50b12-9dbd-4a1f-9697-e12cba91600b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '909137c2da3f40b4bbb1072a22965ada', 'request_id': 'req-cdb50b12-9dbd-4a1f-9697-e12cba91600b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:39.218365 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cdb50b12-9dbd-4a1f-9697-e12cba91600b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '909137c2da3f40b4bbb1072a22965ada', 'request_id': 'req-cdb50b12-9dbd-4a1f-9697-e12cba91600b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:39.229677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cdb50b12-9dbd-4a1f-9697-e12cba91600b admin admin] The token cache is being invalidate because project 909137c2da3f40b4bbb1072a22965ada was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:15:39.233800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cdb50b12-9dbd-4a1f-9697-e12cba91600b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:39.233800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cdb50b12-9dbd-4a1f-9697-e12cba91600b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:39.233800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1406/2820] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:15:38 2022] DELETE /identity/v3/projects/909137c2da3f40b4bbb1072a22965ada => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:15:39.237127 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d24971-c183-4e5e-9419-13502b7ebbe0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:39.237628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d24971-c183-4e5e-9419-13502b7ebbe0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:39.237766 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d24971-c183-4e5e-9419-13502b7ebbe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:39.257700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-60d24971-c183-4e5e-9419-13502b7ebbe0 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:15:39.258907 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-60d24971-c183-4e5e-9419-13502b7ebbe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:39.319139 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1415/2821] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:15:39.327638 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2baa69-02ab-4790-be01-dda673e9f111 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:39.327954 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2baa69-02ab-4790-be01-dda673e9f111 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:39.328230 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2baa69-02ab-4790-be01-dda673e9f111 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:39.347769 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2c2baa69-02ab-4790-be01-dda673e9f111 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:15:39.349008 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2c2baa69-02ab-4790-be01-dda673e9f111 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:39.357241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1407/2822] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:15:39.362144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d24971-c183-4e5e-9419-13502b7ebbe0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:39.372636 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d24971-c183-4e5e-9419-13502b7ebbe0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:39.377124 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2bcc1-a9b1-44fe-8838-b62baaf3ecf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:39.378823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2bcc1-a9b1-44fe-8838-b62baaf3ecf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:39.378823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2bcc1-a9b1-44fe-8838-b62baaf3ecf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:39.378823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a2bcc1-a9b1-44fe-8838-b62baaf3ecf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:39.379104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2bcc1-a9b1-44fe-8838-b62baaf3ecf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:15:39.379264 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2bcc1-a9b1-44fe-8838-b62baaf3ecf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:39.380872 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a2bcc1-a9b1-44fe-8838-b62baaf3ecf8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:39.393073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1416/2823] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:15:39.397729 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2baa69-02ab-4790-be01-dda673e9f111 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:39.407883 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2baa69-02ab-4790-be01-dda673e9f111 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:39.413265 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb80290-71a3-40df-9657-39a5de7fff97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:39.414007 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb80290-71a3-40df-9657-39a5de7fff97 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:39.414282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb80290-71a3-40df-9657-39a5de7fff97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:39.414552 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb80290-71a3-40df-9657-39a5de7fff97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:39.415184 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb80290-71a3-40df-9657-39a5de7fff97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:39.417510 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb80290-71a3-40df-9657-39a5de7fff97 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:39.488242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1408/2824] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:15:39 2022] POST /identity/v3/projects => generated 365 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:15:39.492515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2bcc1-a9b1-44fe-8838-b62baaf3ecf8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:39.503357 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a2bcc1-a9b1-44fe-8838-b62baaf3ecf8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:39.560724 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0503da-c3dd-45e3-a968-100a5354525b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:39.561330 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0503da-c3dd-45e3-a968-100a5354525b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:39.561612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0503da-c3dd-45e3-a968-100a5354525b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:39.561887 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0503da-c3dd-45e3-a968-100a5354525b admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:39.562514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0503da-c3dd-45e3-a968-100a5354525b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:39.564512 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0503da-c3dd-45e3-a968-100a5354525b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:39.654411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1417/2825] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:15:39 2022] POST /identity/v3/users => generated 283 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:15:39.658564 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb80290-71a3-40df-9657-39a5de7fff97 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:39.668905 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb80290-71a3-40df-9657-39a5de7fff97 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:39.673610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e36e6b-fbbb-45f0-9b9e-7bc60b1cb12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:39.674208 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e36e6b-fbbb-45f0-9b9e-7bc60b1cb12f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:39.674490 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e36e6b-fbbb-45f0-9b9e-7bc60b1cb12f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:39.674762 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e36e6b-fbbb-45f0-9b9e-7bc60b1cb12f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:39.675401 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e36e6b-fbbb-45f0-9b9e-7bc60b1cb12f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:39.677032 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e36e6b-fbbb-45f0-9b9e-7bc60b1cb12f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:39.687076 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1409/2826] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:15:39.691179 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0503da-c3dd-45e3-a968-100a5354525b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:39.701630 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0503da-c3dd-45e3-a968-100a5354525b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:39.706182 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfbe54f-1b5a-4724-9ba8-202e4e69d880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:39.706773 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbe54f-1b5a-4724-9ba8-202e4e69d880 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:39.707052 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbe54f-1b5a-4724-9ba8-202e4e69d880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:39.707323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfbe54f-1b5a-4724-9ba8-202e4e69d880 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:39.707981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbe54f-1b5a-4724-9ba8-202e4e69d880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:39.709664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfbe54f-1b5a-4724-9ba8-202e4e69d880 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:39.710884 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:39.721494 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1418/2827] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:15:39.725565 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e36e6b-fbbb-45f0-9b9e-7bc60b1cb12f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:39.735740 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e36e6b-fbbb-45f0-9b9e-7bc60b1cb12f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:39.740313 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815ee25-996a-4f07-a234-b23aeee80ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:39.740950 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815ee25-996a-4f07-a234-b23aeee80ce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:39.741230 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815ee25-996a-4f07-a234-b23aeee80ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:39.741497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7815ee25-996a-4f07-a234-b23aeee80ce2 admin admin] PATH_INFO: `/v3/projects/4f5523541d754be18cb546452bb934b3/users/893fcff399484521a99ec0fd0295e126/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:39.756481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815ee25-996a-4f07-a234-b23aeee80ce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f5523541d754be18cb546452bb934b3, user_id=893fcff399484521a99ec0fd0295e126, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:39.759218 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7815ee25-996a-4f07-a234-b23aeee80ce2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:39.830268 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1410/2828] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:15:39 2022] PUT /identity/v3/projects/4f5523541d754be18cb546452bb934b3/users/893fcff399484521a99ec0fd0295e126/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:15:39.841273 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ed6cd-2127-4ea3-b170-0cd4c688136a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:39.841553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ed6cd-2127-4ea3-b170-0cd4c688136a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:39.841820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ed6cd-2127-4ea3-b170-0cd4c688136a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:39.868861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c22ed6cd-2127-4ea3-b170-0cd4c688136a None None] MFA Rules not processed for user `893fcff399484521a99ec0fd0295e126`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:15:39.870620 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c22ed6cd-2127-4ea3-b170-0cd4c688136a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:39.936078 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1419/2829] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:15:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:15:39.940585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815ee25-996a-4f07-a234-b23aeee80ce2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:40.003673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7815ee25-996a-4f07-a234-b23aeee80ce2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:40.008599 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4ece3-bf92-44da-afd9-aaaac24d2d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:40.009202 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4ece3-bf92-44da-afd9-aaaac24d2d54 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:40.009478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4ece3-bf92-44da-afd9-aaaac24d2d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:40.009749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d4ece3-bf92-44da-afd9-aaaac24d2d54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:40.010404 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4ece3-bf92-44da-afd9-aaaac24d2d54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:40.012356 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d4ece3-bf92-44da-afd9-aaaac24d2d54 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:40.139821 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1411/2830] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:15:39 2022] POST /identity/v3/projects => generated 365 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:15:40.144264 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ed6cd-2127-4ea3-b170-0cd4c688136a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:40.156538 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ed6cd-2127-4ea3-b170-0cd4c688136a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:40.213614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023573b6-c38b-422c-b25e-5900a4735d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:40.214770 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023573b6-c38b-422c-b25e-5900a4735d6d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:40.215099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023573b6-c38b-422c-b25e-5900a4735d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:40.215325 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023573b6-c38b-422c-b25e-5900a4735d6d admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:40.215998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023573b6-c38b-422c-b25e-5900a4735d6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:40.217954 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023573b6-c38b-422c-b25e-5900a4735d6d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:40.292167 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1420/2831] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:15:40 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:15:40.296224 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4ece3-bf92-44da-afd9-aaaac24d2d54 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:40.306598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d4ece3-bf92-44da-afd9-aaaac24d2d54 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:40.311279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe7887-e96b-45cc-b9ec-1ffda44bc213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:40.311934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe7887-e96b-45cc-b9ec-1ffda44bc213 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:40.312212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe7887-e96b-45cc-b9ec-1ffda44bc213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:40.312482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe7887-e96b-45cc-b9ec-1ffda44bc213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:40.313114 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe7887-e96b-45cc-b9ec-1ffda44bc213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:40.314744 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe7887-e96b-45cc-b9ec-1ffda44bc213 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:40.326071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1412/2832] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:15:40.330302 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023573b6-c38b-422c-b25e-5900a4735d6d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:40.341064 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023573b6-c38b-422c-b25e-5900a4735d6d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:40.345598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c93c7-d53b-41b7-a4a5-4b678b8d9499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:40.346252 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c93c7-d53b-41b7-a4a5-4b678b8d9499 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:40.346537 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c93c7-d53b-41b7-a4a5-4b678b8d9499 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:40.346813 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c93c7-d53b-41b7-a4a5-4b678b8d9499 admin admin] PATH_INFO: `/v3/projects/0aee44f2a3614b8cb4c61aaf364c2dfb/users/1231677a9de4401f8d8b099e83f8f6f5/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:40.362363 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c93c7-d53b-41b7-a4a5-4b678b8d9499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aee44f2a3614b8cb4c61aaf364c2dfb, user_id=1231677a9de4401f8d8b099e83f8f6f5, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:40.365269 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c93c7-d53b-41b7-a4a5-4b678b8d9499 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:40.412201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1421/2833] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:15:40 2022] PUT /identity/v3/projects/0aee44f2a3614b8cb4c61aaf364c2dfb/users/1231677a9de4401f8d8b099e83f8f6f5/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:15:40.416602 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe7887-e96b-45cc-b9ec-1ffda44bc213 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:40.427135 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe7887-e96b-45cc-b9ec-1ffda44bc213 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:40.465984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:40.484262 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7db630-deef-44b3-8dfd-6b4b4489e983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:40.484875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7db630-deef-44b3-8dfd-6b4b4489e983 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:40.485162 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7db630-deef-44b3-8dfd-6b4b4489e983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:40.485642 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7db630-deef-44b3-8dfd-6b4b4489e983 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:40.486477 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7db630-deef-44b3-8dfd-6b4b4489e983 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:40.488698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7db630-deef-44b3-8dfd-6b4b4489e983 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:40.498861 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1413/2834] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:15:40.503596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c93c7-d53b-41b7-a4a5-4b678b8d9499 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:40.514092 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c93c7-d53b-41b7-a4a5-4b678b8d9499 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:40.518919 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060de7e4-4b4a-4ee6-9a6d-e49840d72868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:40.521373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060de7e4-4b4a-4ee6-9a6d-e49840d72868 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:40.521373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060de7e4-4b4a-4ee6-9a6d-e49840d72868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:40.521373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060de7e4-4b4a-4ee6-9a6d-e49840d72868 admin admin] PATH_INFO: `/v3/projects/0aee44f2a3614b8cb4c61aaf364c2dfb/users/1231677a9de4401f8d8b099e83f8f6f5/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:40.522352 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060de7e4-4b4a-4ee6-9a6d-e49840d72868 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0aee44f2a3614b8cb4c61aaf364c2dfb, user_id=1231677a9de4401f8d8b099e83f8f6f5, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:40.525021 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060de7e4-4b4a-4ee6-9a6d-e49840d72868 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:40.536511 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1422/2835] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:15:40 2022] PUT /identity/v3/projects/0aee44f2a3614b8cb4c61aaf364c2dfb/users/1231677a9de4401f8d8b099e83f8f6f5/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:15:40.547873 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572002c-74c6-48cc-a564-5f2813f626d3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:40.548166 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572002c-74c6-48cc-a564-5f2813f626d3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:40.550103 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572002c-74c6-48cc-a564-5f2813f626d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:40.576759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e572002c-74c6-48cc-a564-5f2813f626d3 None None] MFA Rules not processed for user `1231677a9de4401f8d8b099e83f8f6f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:15:40.577984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e572002c-74c6-48cc-a564-5f2813f626d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:40.645691 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1414/2836] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:15:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:15:40.657159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060de7e4-4b4a-4ee6-9a6d-e49840d72868 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:40.659096 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-060de7e4-4b4a-4ee6-9a6d-e49840d72868 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:40.704054 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:40.720194 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060de7e4-4b4a-4ee6-9a6d-e49840d72868 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:40.723728 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c37c22-e74d-4ba7-81d1-389ab17df417 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:40.724335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c37c22-e74d-4ba7-81d1-389ab17df417 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:40.724612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c37c22-e74d-4ba7-81d1-389ab17df417 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:40.724896 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c37c22-e74d-4ba7-81d1-389ab17df417 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:40.735304 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c37c22-e74d-4ba7-81d1-389ab17df417 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:40.736774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c37c22-e74d-4ba7-81d1-389ab17df417 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:40.746922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1423/2837] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:15:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:15:40.936178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572002c-74c6-48cc-a564-5f2813f626d3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:40.945849 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e572002c-74c6-48cc-a564-5f2813f626d3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:40.950363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0eeecd-c8ce-46aa-9d8f-3178cd8aeead service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:40.955557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eeecd-c8ce-46aa-9d8f-3178cd8aeead service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:40.955557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eeecd-c8ce-46aa-9d8f-3178cd8aeead service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:40.955557 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0eeecd-c8ce-46aa-9d8f-3178cd8aeead service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:41.013800 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0eeecd-c8ce-46aa-9d8f-3178cd8aeead service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:41.015233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0eeecd-c8ce-46aa-9d8f-3178cd8aeead service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:41.024518 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1415/2838] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:15:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:15:50.749033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=9 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:51.020248 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=73 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:51.839307 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c37c22-e74d-4ba7-81d1-389ab17df417 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:51.904071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c37c22-e74d-4ba7-81d1-389ab17df417 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:51.908759 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398d5a1-cb27-40c6-b16c-b99d8bbdbe3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:51.909183 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398d5a1-cb27-40c6-b16c-b99d8bbdbe3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:51.909483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398d5a1-cb27-40c6-b16c-b99d8bbdbe3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:51.909757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398d5a1-cb27-40c6-b16c-b99d8bbdbe3b admin admin] PATH_INFO: `/v3/users/893fcff399484521a99ec0fd0295e126` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:51.910875 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398d5a1-cb27-40c6-b16c-b99d8bbdbe3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=893fcff399484521a99ec0fd0295e126)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:51.912795 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398d5a1-cb27-40c6-b16c-b99d8bbdbe3b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:52.084082 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a398d5a1-cb27-40c6-b16c-b99d8bbdbe3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '893fcff399484521a99ec0fd0295e126'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:52.149125 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a398d5a1-cb27-40c6-b16c-b99d8bbdbe3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '893fcff399484521a99ec0fd0295e126'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:52.149761 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a398d5a1-cb27-40c6-b16c-b99d8bbdbe3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '893fcff399484521a99ec0fd0295e126'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:52.150384 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a398d5a1-cb27-40c6-b16c-b99d8bbdbe3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '893fcff399484521a99ec0fd0295e126'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:52.163886 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a398d5a1-cb27-40c6-b16c-b99d8bbdbe3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '893fcff399484521a99ec0fd0295e126'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:52.177478 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1424/2839] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:15:51 2022] DELETE /identity/v3/users/893fcff399484521a99ec0fd0295e126 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:15:52.181973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8e6e0-697f-46ee-8a1f-9d2b1395984e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:52.182481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8e6e0-697f-46ee-8a1f-9d2b1395984e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:52.182662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b8e6e0-697f-46ee-8a1f-9d2b1395984e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:52.201688 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-85b8e6e0-697f-46ee-8a1f-9d2b1395984e None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:15:52.202881 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-85b8e6e0-697f-46ee-8a1f-9d2b1395984e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:52.262067 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1416/2840] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:15:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:15:52.279959 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398d5a1-cb27-40c6-b16c-b99d8bbdbe3b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:52.281349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a398d5a1-cb27-40c6-b16c-b99d8bbdbe3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:52.337810 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398d5a1-cb27-40c6-b16c-b99d8bbdbe3b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:52.341880 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14a658-aa88-40ac-8066-fca298111825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:52.342478 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14a658-aa88-40ac-8066-fca298111825 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:52.342756 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14a658-aa88-40ac-8066-fca298111825 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:52.343026 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b14a658-aa88-40ac-8066-fca298111825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:52.352085 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14a658-aa88-40ac-8066-fca298111825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:52.353518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b14a658-aa88-40ac-8066-fca298111825 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:52.362742 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1425/2841] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:15:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:15:52.768969 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8e6e0-697f-46ee-8a1f-9d2b1395984e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:52.770656 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-85b8e6e0-697f-46ee-8a1f-9d2b1395984e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:52.788092 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b8e6e0-697f-46ee-8a1f-9d2b1395984e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:52.794274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:52.794934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:52.795279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:52.795476 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f admin admin] PATH_INFO: `/v3/projects/4f5523541d754be18cb546452bb934b3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:52.796640 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f5523541d754be18cb546452bb934b3)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:52.798537 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:52.893259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f5523541d754be18cb546452bb934b3', 'request_id': 'req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:52.899638 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f5523541d754be18cb546452bb934b3', 'request_id': 'req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:52.960814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f5523541d754be18cb546452bb934b3', 'request_id': 'req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:52.971984 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f admin admin] The token cache is being invalidate because project 4f5523541d754be18cb546452bb934b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:15:52.972275 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:52.972914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:52.974407 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1417/2842] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:15:52 2022] DELETE /identity/v3/projects/4f5523541d754be18cb546452bb934b3 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:15:52.979224 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14a658-aa88-40ac-8066-fca298111825 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:52.989225 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b14a658-aa88-40ac-8066-fca298111825 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:53.007861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:53.046099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4542ce6-7b30-46bd-9a85-63c8a12f1bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:53.046656 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4542ce6-7b30-46bd-9a85-63c8a12f1bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:53.046942 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4542ce6-7b30-46bd-9a85-63c8a12f1bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:53.047213 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4542ce6-7b30-46bd-9a85-63c8a12f1bc3 admin admin] PATH_INFO: `/v3/users/1231677a9de4401f8d8b099e83f8f6f5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:53.048381 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4542ce6-7b30-46bd-9a85-63c8a12f1bc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1231677a9de4401f8d8b099e83f8f6f5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:53.050276 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4542ce6-7b30-46bd-9a85-63c8a12f1bc3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:53.358970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f4542ce6-7b30-46bd-9a85-63c8a12f1bc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1231677a9de4401f8d8b099e83f8f6f5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:53.446508 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f4542ce6-7b30-46bd-9a85-63c8a12f1bc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1231677a9de4401f8d8b099e83f8f6f5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:53.447162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f4542ce6-7b30-46bd-9a85-63c8a12f1bc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1231677a9de4401f8d8b099e83f8f6f5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:53.448677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f4542ce6-7b30-46bd-9a85-63c8a12f1bc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1231677a9de4401f8d8b099e83f8f6f5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:53.461717 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f4542ce6-7b30-46bd-9a85-63c8a12f1bc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1231677a9de4401f8d8b099e83f8f6f5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:53.474335 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1426/2843] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:15:52 2022] DELETE /identity/v3/users/1231677a9de4401f8d8b099e83f8f6f5 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:15:53.877292 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:53.878814 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:53.894232 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc102f8c-c4bb-4591-9a9e-3244cb44fa6f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:53.952004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf69742-2f11-4c11-bc07-61c082b5e2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:53.952629 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf69742-2f11-4c11-bc07-61c082b5e2ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:53.952903 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf69742-2f11-4c11-bc07-61c082b5e2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:53.953180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf69742-2f11-4c11-bc07-61c082b5e2ee admin admin] PATH_INFO: `/v3/projects/0aee44f2a3614b8cb4c61aaf364c2dfb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:53.954280 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf69742-2f11-4c11-bc07-61c082b5e2ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=0aee44f2a3614b8cb4c61aaf364c2dfb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:53.956206 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf69742-2f11-4c11-bc07-61c082b5e2ee admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:54.042662 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:54.059860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6cf69742-2f11-4c11-bc07-61c082b5e2ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0aee44f2a3614b8cb4c61aaf364c2dfb', 'request_id': 'req-6cf69742-2f11-4c11-bc07-61c082b5e2ee'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:54.066375 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6cf69742-2f11-4c11-bc07-61c082b5e2ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0aee44f2a3614b8cb4c61aaf364c2dfb', 'request_id': 'req-6cf69742-2f11-4c11-bc07-61c082b5e2ee'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:54.131990 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6cf69742-2f11-4c11-bc07-61c082b5e2ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0aee44f2a3614b8cb4c61aaf364c2dfb', 'request_id': 'req-6cf69742-2f11-4c11-bc07-61c082b5e2ee'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:54.143177 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6cf69742-2f11-4c11-bc07-61c082b5e2ee admin admin] The token cache is being invalidate because project 0aee44f2a3614b8cb4c61aaf364c2dfb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:15:54.143616 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6cf69742-2f11-4c11-bc07-61c082b5e2ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:54.144783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6cf69742-2f11-4c11-bc07-61c082b5e2ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:15:54.146666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1418/2844] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:15:53 2022] DELETE /identity/v3/projects/0aee44f2a3614b8cb4c61aaf364c2dfb => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:15:54.152282 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39b562-2a2d-4a5d-8824-9e11ed51e6ca None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:54.152893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39b562-2a2d-4a5d-8824-9e11ed51e6ca None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:54.152893 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb39b562-2a2d-4a5d-8824-9e11ed51e6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:54.173589 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fb39b562-2a2d-4a5d-8824-9e11ed51e6ca None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:15:54.174780 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fb39b562-2a2d-4a5d-8824-9e11ed51e6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:54.235119 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1427/2845] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:15:54.242946 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01cdf7-f6e9-41a1-8f23-46e826840ef6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:54.243230 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01cdf7-f6e9-41a1-8f23-46e826840ef6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:54.243524 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb01cdf7-f6e9-41a1-8f23-46e826840ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:54.263164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-bb01cdf7-f6e9-41a1-8f23-46e826840ef6 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:15:54.264403 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bb01cdf7-f6e9-41a1-8f23-46e826840ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:54.272502 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1419/2846] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:15:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:15:54.277277 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39b562-2a2d-4a5d-8824-9e11ed51e6ca None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:54.286826 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb39b562-2a2d-4a5d-8824-9e11ed51e6ca None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:54.291941 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9704fda-37d9-4554-ac01-c66b5296627f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:54.292619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9704fda-37d9-4554-ac01-c66b5296627f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:54.292898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9704fda-37d9-4554-ac01-c66b5296627f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:54.293168 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9704fda-37d9-4554-ac01-c66b5296627f admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:54.293808 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9704fda-37d9-4554-ac01-c66b5296627f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:15:54.294103 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9704fda-37d9-4554-ac01-c66b5296627f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:54.295700 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9704fda-37d9-4554-ac01-c66b5296627f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:54.307456 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1428/2847] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:15:54.312167 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01cdf7-f6e9-41a1-8f23-46e826840ef6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:54.321477 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb01cdf7-f6e9-41a1-8f23-46e826840ef6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:54.327214 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2d6c8-4179-47b8-bd47-4879bc7b9681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:54.327860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2d6c8-4179-47b8-bd47-4879bc7b9681 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:54.328153 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2d6c8-4179-47b8-bd47-4879bc7b9681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:54.328421 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a2d6c8-4179-47b8-bd47-4879bc7b9681 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:54.329044 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a2d6c8-4179-47b8-bd47-4879bc7b9681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:54.330953 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a2d6c8-4179-47b8-bd47-4879bc7b9681 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:54.413556 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1420/2848] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:15:54 2022] POST /identity/v3/projects => generated 391 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:15:54.421483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9704fda-37d9-4554-ac01-c66b5296627f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:54.431619 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9704fda-37d9-4554-ac01-c66b5296627f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:54.458390 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:54.488957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f9202-b2c7-4563-8124-b277c19fca77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:54.489576 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f9202-b2c7-4563-8124-b277c19fca77 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:54.489836 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f9202-b2c7-4563-8124-b277c19fca77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:54.490104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f9202-b2c7-4563-8124-b277c19fca77 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:54.490718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f9202-b2c7-4563-8124-b277c19fca77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:54.492654 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f9202-b2c7-4563-8124-b277c19fca77 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:54.584301 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1429/2849] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:15:54 2022] POST /identity/v3/users => generated 296 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:15:54.588678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2d6c8-4179-47b8-bd47-4879bc7b9681 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:54.599212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a2d6c8-4179-47b8-bd47-4879bc7b9681 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:54.604388 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700b75c-9dc3-49c8-af8a-c069503fbfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:54.605344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700b75c-9dc3-49c8-af8a-c069503fbfe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:54.605624 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700b75c-9dc3-49c8-af8a-c069503fbfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:54.605901 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e700b75c-9dc3-49c8-af8a-c069503fbfe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:54.606543 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e700b75c-9dc3-49c8-af8a-c069503fbfe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:54.608212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e700b75c-9dc3-49c8-af8a-c069503fbfe1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:54.618216 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1421/2850] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:15:54.622806 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f9202-b2c7-4563-8124-b277c19fca77 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:54.632687 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f9202-b2c7-4563-8124-b277c19fca77 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:54.638061 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727572aa-3331-477d-9f48-efb6b8e80aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:54.638663 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727572aa-3331-477d-9f48-efb6b8e80aff admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:54.638951 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727572aa-3331-477d-9f48-efb6b8e80aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:54.639229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727572aa-3331-477d-9f48-efb6b8e80aff admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:54.639909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727572aa-3331-477d-9f48-efb6b8e80aff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:54.641532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727572aa-3331-477d-9f48-efb6b8e80aff admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:54.653050 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1430/2851] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:15:54.657242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700b75c-9dc3-49c8-af8a-c069503fbfe1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:54.667517 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e700b75c-9dc3-49c8-af8a-c069503fbfe1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:54.672063 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947ee91-958a-4f09-860a-fe038e5b43c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:54.672702 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947ee91-958a-4f09-860a-fe038e5b43c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:54.672982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947ee91-958a-4f09-860a-fe038e5b43c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:54.673259 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3947ee91-958a-4f09-860a-fe038e5b43c3 admin admin] PATH_INFO: `/v3/projects/ab6c2b2bed5547b3a2b971b049419612/users/9677940b58764cc48f2b5d03cc1fac94/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:54.688164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947ee91-958a-4f09-860a-fe038e5b43c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab6c2b2bed5547b3a2b971b049419612, user_id=9677940b58764cc48f2b5d03cc1fac94, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:54.690818 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3947ee91-958a-4f09-860a-fe038e5b43c3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:54.783939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1422/2852] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:15:54 2022] PUT /identity/v3/projects/ab6c2b2bed5547b3a2b971b049419612/users/9677940b58764cc48f2b5d03cc1fac94/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:15:54.801807 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f226b7-f364-4045-9c7f-4263e226f607 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:54.802098 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f226b7-f364-4045-9c7f-4263e226f607 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:54.802377 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f226b7-f364-4045-9c7f-4263e226f607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:54.831441 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b2f226b7-f364-4045-9c7f-4263e226f607 None None] MFA Rules not processed for user `9677940b58764cc48f2b5d03cc1fac94`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:15:54.832716 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b2f226b7-f364-4045-9c7f-4263e226f607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:54.898691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1431/2853] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:15:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:15:54.903938 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947ee91-958a-4f09-860a-fe038e5b43c3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:54.966258 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3947ee91-958a-4f09-860a-fe038e5b43c3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:54.970850 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e3ec2-9874-4c2d-afcf-8127e22671ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:54.971481 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e3ec2-9874-4c2d-afcf-8127e22671ac admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:54.971802 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e3ec2-9874-4c2d-afcf-8127e22671ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:54.972078 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e3ec2-9874-4c2d-afcf-8127e22671ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:54.972716 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e3ec2-9874-4c2d-afcf-8127e22671ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:54.974622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126e3ec2-9874-4c2d-afcf-8127e22671ac admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:55.099598 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1423/2854] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:15:54 2022] POST /identity/v3/projects => generated 391 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:15:55.104540 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f226b7-f364-4045-9c7f-4263e226f607 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:55.115144 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f226b7-f364-4045-9c7f-4263e226f607 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:55.171145 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498f473-45df-47e7-8a2b-5e7dd60f228d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:55.171840 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498f473-45df-47e7-8a2b-5e7dd60f228d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:55.172048 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498f473-45df-47e7-8a2b-5e7dd60f228d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:55.172319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498f473-45df-47e7-8a2b-5e7dd60f228d admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:55.172944 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498f473-45df-47e7-8a2b-5e7dd60f228d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:55.174858 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498f473-45df-47e7-8a2b-5e7dd60f228d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:55.257820 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1432/2855] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:15:55 2022] POST /identity/v3/users => generated 302 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:15:55.262460 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e3ec2-9874-4c2d-afcf-8127e22671ac admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:55.273868 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e3ec2-9874-4c2d-afcf-8127e22671ac admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:55.279601 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08871f1-3407-4738-93c2-032732919978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:55.280233 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08871f1-3407-4738-93c2-032732919978 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:55.280515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08871f1-3407-4738-93c2-032732919978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:55.280786 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08871f1-3407-4738-93c2-032732919978 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:55.281424 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08871f1-3407-4738-93c2-032732919978 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:55.283155 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08871f1-3407-4738-93c2-032732919978 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:55.294369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1424/2856] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:15:55.298598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498f473-45df-47e7-8a2b-5e7dd60f228d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:55.309253 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498f473-45df-47e7-8a2b-5e7dd60f228d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:55.313844 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3692bcd-5bd0-4e77-b494-da8b06bfe1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:55.314495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3692bcd-5bd0-4e77-b494-da8b06bfe1e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:55.314778 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3692bcd-5bd0-4e77-b494-da8b06bfe1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:55.315055 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3692bcd-5bd0-4e77-b494-da8b06bfe1e8 admin admin] PATH_INFO: `/v3/projects/5271c330ef284e02b851ac99c2628e4d/users/d7d940eea0fb4beba38580f5d94ea31c/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:55.330534 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3692bcd-5bd0-4e77-b494-da8b06bfe1e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5271c330ef284e02b851ac99c2628e4d, user_id=d7d940eea0fb4beba38580f5d94ea31c, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:55.333222 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3692bcd-5bd0-4e77-b494-da8b06bfe1e8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:55.435450 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1433/2857] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:15:55 2022] PUT /identity/v3/projects/5271c330ef284e02b851ac99c2628e4d/users/d7d940eea0fb4beba38580f5d94ea31c/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:15:55.439430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08871f1-3407-4738-93c2-032732919978 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:55.449953 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08871f1-3407-4738-93c2-032732919978 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:55.470361 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:55.506467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3a82bc-34d0-45d5-a306-446d1bd59fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:55.507069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a82bc-34d0-45d5-a306-446d1bd59fde admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:55.507344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a82bc-34d0-45d5-a306-446d1bd59fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:55.507612 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3a82bc-34d0-45d5-a306-446d1bd59fde admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:55.508331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a82bc-34d0-45d5-a306-446d1bd59fde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:55.510666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3a82bc-34d0-45d5-a306-446d1bd59fde admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:55.520432 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1425/2858] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:15:55.524922 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3692bcd-5bd0-4e77-b494-da8b06bfe1e8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:55.531135 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:15:55.536589 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3692bcd-5bd0-4e77-b494-da8b06bfe1e8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:55.541307 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473fc84-5224-410a-9b3d-28f903a9f207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:55.542071 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473fc84-5224-410a-9b3d-28f903a9f207 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:55.542349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473fc84-5224-410a-9b3d-28f903a9f207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:55.542622 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3473fc84-5224-410a-9b3d-28f903a9f207 admin admin] PATH_INFO: `/v3/projects/5271c330ef284e02b851ac99c2628e4d/users/d7d940eea0fb4beba38580f5d94ea31c/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:55.544553 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473fc84-5224-410a-9b3d-28f903a9f207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5271c330ef284e02b851ac99c2628e4d, user_id=d7d940eea0fb4beba38580f5d94ea31c, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:55.547191 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3473fc84-5224-410a-9b3d-28f903a9f207 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:55.557583 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1434/2859] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:15:55 2022] PUT /identity/v3/projects/5271c330ef284e02b851ac99c2628e4d/users/d7d940eea0fb4beba38580f5d94ea31c/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:15:55.625312 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9757de6-5795-4936-9019-855049816273 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:55.625653 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9757de6-5795-4936-9019-855049816273 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:55.625933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9757de6-5795-4936-9019-855049816273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:55.656672 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b9757de6-5795-4936-9019-855049816273 None None] MFA Rules not processed for user `d7d940eea0fb4beba38580f5d94ea31c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:15:55.657922 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b9757de6-5795-4936-9019-855049816273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:55.725423 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1426/2860] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:15:55 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:15:55.736946 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473fc84-5224-410a-9b3d-28f903a9f207 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:55.738648 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3473fc84-5224-410a-9b3d-28f903a9f207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:55.796514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3473fc84-5224-410a-9b3d-28f903a9f207 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:55.800064 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e90fe-ddf2-498f-bb4d-3ce2c758881b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:55.800660 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e90fe-ddf2-498f-bb4d-3ce2c758881b service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:55.800940 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e90fe-ddf2-498f-bb4d-3ce2c758881b service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:55.801216 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4e90fe-ddf2-498f-bb4d-3ce2c758881b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:55.810650 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e90fe-ddf2-498f-bb4d-3ce2c758881b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:55.812104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4e90fe-ddf2-498f-bb4d-3ce2c758881b service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:55.821429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1435/2861] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:15:55 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:15:55.870789 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9757de6-5795-4936-9019-855049816273 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:15:55.872515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b9757de6-5795-4936-9019-855049816273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:15:55.889884 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9757de6-5795-4936-9019-855049816273 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:15:55.894997 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4ffd6a-5ea1-4508-8066-18036aa1b404 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:15:55.895599 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ffd6a-5ea1-4508-8066-18036aa1b404 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:15:55.895906 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ffd6a-5ea1-4508-8066-18036aa1b404 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:15:55.896443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4ffd6a-5ea1-4508-8066-18036aa1b404 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:15:55.952580 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ffd6a-5ea1-4508-8066-18036aa1b404 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:15:55.954004 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4ffd6a-5ea1-4508-8066-18036aa1b404 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:15:55.962721 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1427/2862] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:15:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:16:00.611209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e90fe-ddf2-498f-bb4d-3ce2c758881b service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:00.675525 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4e90fe-ddf2-498f-bb4d-3ce2c758881b service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:00.680064 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f05fdc-08fd-46fc-9d59-a4ae4d9eca88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:00.680692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f05fdc-08fd-46fc-9d59-a4ae4d9eca88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:00.680973 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f05fdc-08fd-46fc-9d59-a4ae4d9eca88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:00.681248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f05fdc-08fd-46fc-9d59-a4ae4d9eca88 admin admin] PATH_INFO: `/v3/users/9677940b58764cc48f2b5d03cc1fac94` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:00.683129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f05fdc-08fd-46fc-9d59-a4ae4d9eca88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9677940b58764cc48f2b5d03cc1fac94)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:00.685058 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f05fdc-08fd-46fc-9d59-a4ae4d9eca88 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:00.808726 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e7f05fdc-08fd-46fc-9d59-a4ae4d9eca88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9677940b58764cc48f2b5d03cc1fac94'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:00.852104 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e7f05fdc-08fd-46fc-9d59-a4ae4d9eca88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9677940b58764cc48f2b5d03cc1fac94'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:00.852738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e7f05fdc-08fd-46fc-9d59-a4ae4d9eca88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9677940b58764cc48f2b5d03cc1fac94'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:00.853362 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e7f05fdc-08fd-46fc-9d59-a4ae4d9eca88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9677940b58764cc48f2b5d03cc1fac94'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:00.866248 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e7f05fdc-08fd-46fc-9d59-a4ae4d9eca88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9677940b58764cc48f2b5d03cc1fac94'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:00.879328 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1436/2863] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:16:00 2022] DELETE /identity/v3/users/9677940b58764cc48f2b5d03cc1fac94 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:16:00.883811 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64879395-6526-48d1-8993-83a80a31a579 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:00.884281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64879395-6526-48d1-8993-83a80a31a579 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:00.884379 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64879395-6526-48d1-8993-83a80a31a579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:00.912783 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-64879395-6526-48d1-8993-83a80a31a579 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:16:00.914006 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-64879395-6526-48d1-8993-83a80a31a579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:00.959338 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:00.978426 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1428/2864] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:16:00.988637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f05fdc-08fd-46fc-9d59-a4ae4d9eca88 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:00.990143 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e7f05fdc-08fd-46fc-9d59-a4ae4d9eca88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:01.004822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:01.049704 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f05fdc-08fd-46fc-9d59-a4ae4d9eca88 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:01.053964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b395a48-ca25-4c72-9189-1aa329ec47af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:01.054568 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b395a48-ca25-4c72-9189-1aa329ec47af service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:01.054845 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b395a48-ca25-4c72-9189-1aa329ec47af service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:01.055115 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b395a48-ca25-4c72-9189-1aa329ec47af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:01.065047 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b395a48-ca25-4c72-9189-1aa329ec47af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:01.066382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b395a48-ca25-4c72-9189-1aa329ec47af service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:01.076646 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1437/2865] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:16:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:16:01.574304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64879395-6526-48d1-8993-83a80a31a579 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:01.575831 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-64879395-6526-48d1-8993-83a80a31a579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:01.591520 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64879395-6526-48d1-8993-83a80a31a579 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:01.599677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b3e0f-e393-411d-b60c-62aab510dbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:01.599677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b3e0f-e393-411d-b60c-62aab510dbfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:01.599677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b3e0f-e393-411d-b60c-62aab510dbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:01.599677 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389b3e0f-e393-411d-b60c-62aab510dbfa admin admin] PATH_INFO: `/v3/projects/ab6c2b2bed5547b3a2b971b049419612` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:01.600566 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b3e0f-e393-411d-b60c-62aab510dbfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab6c2b2bed5547b3a2b971b049419612)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:01.601346 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389b3e0f-e393-411d-b60c-62aab510dbfa admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:01.722416 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-389b3e0f-e393-411d-b60c-62aab510dbfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab6c2b2bed5547b3a2b971b049419612', 'request_id': 'req-389b3e0f-e393-411d-b60c-62aab510dbfa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:01.729460 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-389b3e0f-e393-411d-b60c-62aab510dbfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab6c2b2bed5547b3a2b971b049419612', 'request_id': 'req-389b3e0f-e393-411d-b60c-62aab510dbfa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:01.783703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-389b3e0f-e393-411d-b60c-62aab510dbfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab6c2b2bed5547b3a2b971b049419612', 'request_id': 'req-389b3e0f-e393-411d-b60c-62aab510dbfa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:01.795530 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-389b3e0f-e393-411d-b60c-62aab510dbfa admin admin] The token cache is being invalidate because project ab6c2b2bed5547b3a2b971b049419612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:16:01.795853 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-389b3e0f-e393-411d-b60c-62aab510dbfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:01.796488 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-389b3e0f-e393-411d-b60c-62aab510dbfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:01.797974 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1429/2866] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:16:01 2022] DELETE /identity/v3/projects/ab6c2b2bed5547b3a2b971b049419612 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:16:01.802427 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b395a48-ca25-4c72-9189-1aa329ec47af service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:01.812409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b395a48-ca25-4c72-9189-1aa329ec47af service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:01.869428 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17796f6-aa03-4f6e-aa16-ce96997079e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:01.870073 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17796f6-aa03-4f6e-aa16-ce96997079e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:01.870349 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17796f6-aa03-4f6e-aa16-ce96997079e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:01.870621 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17796f6-aa03-4f6e-aa16-ce96997079e7 admin admin] PATH_INFO: `/v3/users/d7d940eea0fb4beba38580f5d94ea31c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:01.871796 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17796f6-aa03-4f6e-aa16-ce96997079e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7d940eea0fb4beba38580f5d94ea31c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:01.873760 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17796f6-aa03-4f6e-aa16-ce96997079e7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:02.067681 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c17796f6-aa03-4f6e-aa16-ce96997079e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7d940eea0fb4beba38580f5d94ea31c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:02.143414 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c17796f6-aa03-4f6e-aa16-ce96997079e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7d940eea0fb4beba38580f5d94ea31c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:02.143999 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c17796f6-aa03-4f6e-aa16-ce96997079e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7d940eea0fb4beba38580f5d94ea31c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:02.144645 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c17796f6-aa03-4f6e-aa16-ce96997079e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7d940eea0fb4beba38580f5d94ea31c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:02.158021 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c17796f6-aa03-4f6e-aa16-ce96997079e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7d940eea0fb4beba38580f5d94ea31c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:02.170185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1438/2867] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:16:01 2022] DELETE /identity/v3/users/d7d940eea0fb4beba38580f5d94ea31c => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:16:02.550856 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b3e0f-e393-411d-b60c-62aab510dbfa admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:02.554759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-389b3e0f-e393-411d-b60c-62aab510dbfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:02.568357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389b3e0f-e393-411d-b60c-62aab510dbfa admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:02.627057 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db565d92-4cd5-468a-a4b7-6198ee560aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:02.627737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db565d92-4cd5-468a-a4b7-6198ee560aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:02.627987 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db565d92-4cd5-468a-a4b7-6198ee560aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:02.628240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db565d92-4cd5-468a-a4b7-6198ee560aee admin admin] PATH_INFO: `/v3/projects/5271c330ef284e02b851ac99c2628e4d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:02.629354 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db565d92-4cd5-468a-a4b7-6198ee560aee admin admin] RBAC: Authorizing `identity:delete_project(project_id=5271c330ef284e02b851ac99c2628e4d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:02.631241 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db565d92-4cd5-468a-a4b7-6198ee560aee admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:02.743966 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-db565d92-4cd5-468a-a4b7-6198ee560aee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5271c330ef284e02b851ac99c2628e4d', 'request_id': 'req-db565d92-4cd5-468a-a4b7-6198ee560aee'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:02.750848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-db565d92-4cd5-468a-a4b7-6198ee560aee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5271c330ef284e02b851ac99c2628e4d', 'request_id': 'req-db565d92-4cd5-468a-a4b7-6198ee560aee'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:02.823311 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-db565d92-4cd5-468a-a4b7-6198ee560aee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5271c330ef284e02b851ac99c2628e4d', 'request_id': 'req-db565d92-4cd5-468a-a4b7-6198ee560aee'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:02.830193 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:02.834845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-db565d92-4cd5-468a-a4b7-6198ee560aee admin admin] The token cache is being invalidate because project 5271c330ef284e02b851ac99c2628e4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:16:02.835137 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-db565d92-4cd5-468a-a4b7-6198ee560aee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:02.835810 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-db565d92-4cd5-468a-a4b7-6198ee560aee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:02.837312 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1430/2868] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:16:02 2022] DELETE /identity/v3/projects/5271c330ef284e02b851ac99c2628e4d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:16:02.843583 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc62ef5-6a59-477b-9394-cbc6068bc5b4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:02.844178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc62ef5-6a59-477b-9394-cbc6068bc5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:02.844178 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc62ef5-6a59-477b-9394-cbc6068bc5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:02.864013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fbc62ef5-6a59-477b-9394-cbc6068bc5b4 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:16:02.865218 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fbc62ef5-6a59-477b-9394-cbc6068bc5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:02.894294 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:02.926231 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1439/2869] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 26 18:16:02.934655 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3c886-4de3-4da3-8f3f-fdb84b89fe74 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:02.934938 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3c886-4de3-4da3-8f3f-fdb84b89fe74 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:02.935213 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3c886-4de3-4da3-8f3f-fdb84b89fe74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:02.955083 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-86b3c886-4de3-4da3-8f3f-fdb84b89fe74 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:16:02.956344 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-86b3c886-4de3-4da3-8f3f-fdb84b89fe74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:02.964693 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1431/2870] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:16:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:16:02.968928 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc62ef5-6a59-477b-9394-cbc6068bc5b4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:02.981170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc62ef5-6a59-477b-9394-cbc6068bc5b4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:02.985918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb4eb5-c95f-4b0a-9d19-fd46eb7d7bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:02.986865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb4eb5-c95f-4b0a-9d19-fd46eb7d7bdd admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:02.986865 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb4eb5-c95f-4b0a-9d19-fd46eb7d7bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:02.988009 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb4eb5-c95f-4b0a-9d19-fd46eb7d7bdd admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:02.988009 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb4eb5-c95f-4b0a-9d19-fd46eb7d7bdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:16:02.988223 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb4eb5-c95f-4b0a-9d19-fd46eb7d7bdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:02.989779 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb4eb5-c95f-4b0a-9d19-fd46eb7d7bdd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:03.002826 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1440/2871] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:16:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:16:03.007188 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3c886-4de3-4da3-8f3f-fdb84b89fe74 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:03.016485 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3c886-4de3-4da3-8f3f-fdb84b89fe74 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:03.026277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adb9b7-12e2-4557-b54b-632883f21b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:03.026277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adb9b7-12e2-4557-b54b-632883f21b31 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:03.026277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adb9b7-12e2-4557-b54b-632883f21b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:03.026277 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adb9b7-12e2-4557-b54b-632883f21b31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:03.026686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07adb9b7-12e2-4557-b54b-632883f21b31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:03.026686 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07adb9b7-12e2-4557-b54b-632883f21b31 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:03.157400 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1432/2872] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:16:03 2022] POST /identity/v3/projects => generated 375 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:16:03.161563 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb4eb5-c95f-4b0a-9d19-fd46eb7d7bdd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:03.172514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb4eb5-c95f-4b0a-9d19-fd46eb7d7bdd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:03.230788 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899b1dd-e992-4a8f-a6aa-f295789bd33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:03.231404 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899b1dd-e992-4a8f-a6aa-f295789bd33a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:03.231729 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899b1dd-e992-4a8f-a6aa-f295789bd33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:03.232012 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b899b1dd-e992-4a8f-a6aa-f295789bd33a admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:03.232655 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899b1dd-e992-4a8f-a6aa-f295789bd33a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:03.234614 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b899b1dd-e992-4a8f-a6aa-f295789bd33a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:03.355722 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1441/2873] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:16:03 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:16:03.359799 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adb9b7-12e2-4557-b54b-632883f21b31 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:03.370698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adb9b7-12e2-4557-b54b-632883f21b31 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:03.375330 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afee2e-c89e-4e8e-91c0-42429f502bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:03.375956 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afee2e-c89e-4e8e-91c0-42429f502bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:03.376236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afee2e-c89e-4e8e-91c0-42429f502bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:03.376501 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afee2e-c89e-4e8e-91c0-42429f502bf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:03.377132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71afee2e-c89e-4e8e-91c0-42429f502bf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:03.378726 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71afee2e-c89e-4e8e-91c0-42429f502bf8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:03.389533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1433/2874] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:16:03.393714 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899b1dd-e992-4a8f-a6aa-f295789bd33a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:03.404112 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b899b1dd-e992-4a8f-a6aa-f295789bd33a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:03.408892 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30ced2-f167-444d-a895-2a9103688968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:03.409488 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30ced2-f167-444d-a895-2a9103688968 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:03.409772 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30ced2-f167-444d-a895-2a9103688968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:03.410042 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30ced2-f167-444d-a895-2a9103688968 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:03.410693 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30ced2-f167-444d-a895-2a9103688968 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:03.412333 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30ced2-f167-444d-a895-2a9103688968 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:03.423518 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1442/2875] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:16:03.427406 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afee2e-c89e-4e8e-91c0-42429f502bf8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:03.438169 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afee2e-c89e-4e8e-91c0-42429f502bf8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:03.442737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011a146-ca94-4a76-b593-c046e2edcd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:03.443409 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011a146-ca94-4a76-b593-c046e2edcd93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:03.443750 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011a146-ca94-4a76-b593-c046e2edcd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:03.444025 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011a146-ca94-4a76-b593-c046e2edcd93 admin admin] PATH_INFO: `/v3/projects/0107afe7c22643be9730dc52deb95dad/users/03aceec0c7744a43b4319e0354b958cc/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:03.459581 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011a146-ca94-4a76-b593-c046e2edcd93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0107afe7c22643be9730dc52deb95dad, user_id=03aceec0c7744a43b4319e0354b958cc, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:03.462282 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1011a146-ca94-4a76-b593-c046e2edcd93 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:03.537832 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1434/2876] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:16:03 2022] PUT /identity/v3/projects/0107afe7c22643be9730dc52deb95dad/users/03aceec0c7744a43b4319e0354b958cc/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 26 18:16:03.548664 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb969be-cfbb-4254-8158-0ed660198452 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:03.548952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb969be-cfbb-4254-8158-0ed660198452 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:03.549229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb969be-cfbb-4254-8158-0ed660198452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:03.577418 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4cb969be-cfbb-4254-8158-0ed660198452 None None] MFA Rules not processed for user `03aceec0c7744a43b4319e0354b958cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:16:03.578634 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4cb969be-cfbb-4254-8158-0ed660198452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:03.642966 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1443/2877] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:16:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:16:03.654943 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011a146-ca94-4a76-b593-c046e2edcd93 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:03.656788 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1011a146-ca94-4a76-b593-c046e2edcd93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:03.712932 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1011a146-ca94-4a76-b593-c046e2edcd93 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:03.717290 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50e4243-9a6e-416b-8fdc-30635e74ef12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:03.717885 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e4243-9a6e-416b-8fdc-30635e74ef12 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:03.718162 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e4243-9a6e-416b-8fdc-30635e74ef12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:03.718429 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50e4243-9a6e-416b-8fdc-30635e74ef12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:03.726665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e4243-9a6e-416b-8fdc-30635e74ef12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:03.728115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50e4243-9a6e-416b-8fdc-30635e74ef12 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:03.736963 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1435/2878] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:16:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:16:10.835612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb969be-cfbb-4254-8158-0ed660198452 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:10.841902 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4cb969be-cfbb-4254-8158-0ed660198452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:10.875253 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:10.922367 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb969be-cfbb-4254-8158-0ed660198452 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:10.933793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211aa3f0-d536-4d14-b48a-fc2a0a7c351a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:10.933793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211aa3f0-d536-4d14-b48a-fc2a0a7c351a service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:10.933793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211aa3f0-d536-4d14-b48a-fc2a0a7c351a service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:10.933793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211aa3f0-d536-4d14-b48a-fc2a0a7c351a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:10.934199 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-211aa3f0-d536-4d14-b48a-fc2a0a7c351a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:10.948036 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211aa3f0-d536-4d14-b48a-fc2a0a7c351a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:10.949775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211aa3f0-d536-4d14-b48a-fc2a0a7c351a service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:10.963908 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1444/2879] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:16:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:16:13.733101 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:21.247146 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=21 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:24.441826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474071ff-e5fd-489f-9a41-d0df4b71f34c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:24.442355 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474071ff-e5fd-489f-9a41-d0df4b71f34c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:24.446351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474071ff-e5fd-489f-9a41-d0df4b71f34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:24.465474 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-474071ff-e5fd-489f-9a41-d0df4b71f34c None None] MFA Rules not processed for user `1d201727ffae42c4bd704d26c03c5d9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:16:24.466807 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-474071ff-e5fd-489f-9a41-d0df4b71f34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:24.526609 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1436/2880] 10.222.0.67 () {58 vars in 967 bytes} [Mon Sep 26 18:16:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:16:24.573549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211aa3f0-d536-4d14-b48a-fc2a0a7c351a service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:24.574981 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-211aa3f0-d536-4d14-b48a-fc2a0a7c351a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:24.647521 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211aa3f0-d536-4d14-b48a-fc2a0a7c351a service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:24.653998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbc566-8c56-4f77-ad30-ef79af2d8d34 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ba12b94717a4a40b55635192840d3c3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:24.654609 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbc566-8c56-4f77-ad30-ef79af2d8d34 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:24.654889 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbc566-8c56-4f77-ad30-ef79af2d8d34 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:24.655165 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbc566-8c56-4f77-ad30-ef79af2d8d34 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:24.663825 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbc566-8c56-4f77-ad30-ef79af2d8d34 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:24.665432 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbc566-8c56-4f77-ad30-ef79af2d8d34 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:24.673591 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1445/2881] 10.222.0.67 () {60 vars in 1384 bytes} [Mon Sep 26 18:16:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 26 18:16:28.629334 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474071ff-e5fd-489f-9a41-d0df4b71f34c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:28.696117 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474071ff-e5fd-489f-9a41-d0df4b71f34c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:28.700967 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7d304-49a7-4bdf-9e61-0becebd4e5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:28.701709 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7d304-49a7-4bdf-9e61-0becebd4e5cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:28.716390 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7d304-49a7-4bdf-9e61-0becebd4e5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:28.716390 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e7d304-49a7-4bdf-9e61-0becebd4e5cd admin admin] PATH_INFO: `/v3/users/03aceec0c7744a43b4319e0354b958cc` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:28.716390 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7d304-49a7-4bdf-9e61-0becebd4e5cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=03aceec0c7744a43b4319e0354b958cc)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:28.716390 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e7d304-49a7-4bdf-9e61-0becebd4e5cd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:28.915439 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c8e7d304-49a7-4bdf-9e61-0becebd4e5cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03aceec0c7744a43b4319e0354b958cc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:28.989294 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c8e7d304-49a7-4bdf-9e61-0becebd4e5cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03aceec0c7744a43b4319e0354b958cc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:28.990326 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c8e7d304-49a7-4bdf-9e61-0becebd4e5cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03aceec0c7744a43b4319e0354b958cc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:28.991105 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c8e7d304-49a7-4bdf-9e61-0becebd4e5cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03aceec0c7744a43b4319e0354b958cc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:29.028168 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c8e7d304-49a7-4bdf-9e61-0becebd4e5cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03aceec0c7744a43b4319e0354b958cc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:29.057826 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1437/2882] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:16:28 2022] DELETE /identity/v3/users/03aceec0c7744a43b4319e0354b958cc => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:16:29.062545 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837136dd-a5bd-412b-9f3a-f1239cbfe11e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:29.063054 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837136dd-a5bd-412b-9f3a-f1239cbfe11e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:29.063159 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837136dd-a5bd-412b-9f3a-f1239cbfe11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:29.089171 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-837136dd-a5bd-412b-9f3a-f1239cbfe11e None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:16:29.090556 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-837136dd-a5bd-412b-9f3a-f1239cbfe11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:29.170852 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1446/2883] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:16:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:16:29.182934 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7d304-49a7-4bdf-9e61-0becebd4e5cd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:29.184443 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c8e7d304-49a7-4bdf-9e61-0becebd4e5cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:29.234502 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:29.281430 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e7d304-49a7-4bdf-9e61-0becebd4e5cd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:29.286126 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81792a9b-c40c-48d4-b2c1-3c97b50f5c99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:29.286887 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81792a9b-c40c-48d4-b2c1-3c97b50f5c99 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:29.287182 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81792a9b-c40c-48d4-b2c1-3c97b50f5c99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:29.287434 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81792a9b-c40c-48d4-b2c1-3c97b50f5c99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:29.298202 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81792a9b-c40c-48d4-b2c1-3c97b50f5c99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:29.299812 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81792a9b-c40c-48d4-b2c1-3c97b50f5c99 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:29.310554 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1438/2884] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:16:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:16:29.659392 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837136dd-a5bd-412b-9f3a-f1239cbfe11e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:29.660863 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-837136dd-a5bd-412b-9f3a-f1239cbfe11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:29.737752 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837136dd-a5bd-412b-9f3a-f1239cbfe11e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:29.743280 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ce9e0d-aefb-4e4d-b88a-b6d24da8b073 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:29.748408 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce9e0d-aefb-4e4d-b88a-b6d24da8b073 service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:29.748691 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce9e0d-aefb-4e4d-b88a-b6d24da8b073 service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:29.748964 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ce9e0d-aefb-4e4d-b88a-b6d24da8b073 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:29.750982 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-17ce9e0d-aefb-4e4d-b88a-b6d24da8b073 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:29.763584 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce9e0d-aefb-4e4d-b88a-b6d24da8b073 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:29.765201 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ce9e0d-aefb-4e4d-b88a-b6d24da8b073 service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:29.775622 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1447/2885] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:16:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:16:29.845652 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81792a9b-c40c-48d4-b2c1-3c97b50f5c99 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:29.848229 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-81792a9b-c40c-48d4-b2c1-3c97b50f5c99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:29.863152 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81792a9b-c40c-48d4-b2c1-3c97b50f5c99 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:29.868497 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6339937-c968-4c0d-a80f-7894d478c8fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:29.869575 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6339937-c968-4c0d-a80f-7894d478c8fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:29.870094 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6339937-c968-4c0d-a80f-7894d478c8fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:29.870444 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6339937-c968-4c0d-a80f-7894d478c8fa admin admin] PATH_INFO: `/v3/projects/0107afe7c22643be9730dc52deb95dad` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:29.871865 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6339937-c968-4c0d-a80f-7894d478c8fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0107afe7c22643be9730dc52deb95dad)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:29.874142 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6339937-c968-4c0d-a80f-7894d478c8fa admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:30.000074 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6339937-c968-4c0d-a80f-7894d478c8fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0107afe7c22643be9730dc52deb95dad', 'request_id': 'req-b6339937-c968-4c0d-a80f-7894d478c8fa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:30.006721 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6339937-c968-4c0d-a80f-7894d478c8fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0107afe7c22643be9730dc52deb95dad', 'request_id': 'req-b6339937-c968-4c0d-a80f-7894d478c8fa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:30.063125 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6339937-c968-4c0d-a80f-7894d478c8fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0107afe7c22643be9730dc52deb95dad', 'request_id': 'req-b6339937-c968-4c0d-a80f-7894d478c8fa'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:30.077690 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6339937-c968-4c0d-a80f-7894d478c8fa admin admin] The token cache is being invalidate because project 0107afe7c22643be9730dc52deb95dad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:16:30.077985 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6339937-c968-4c0d-a80f-7894d478c8fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:30.080132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6339937-c968-4c0d-a80f-7894d478c8fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:30.082110 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1439/2886] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:16:29 2022] DELETE /identity/v3/projects/0107afe7c22643be9730dc52deb95dad => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:16:30.088214 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a804a-e145-4ca5-ba8a-02d316139cba None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:30.088823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a804a-e145-4ca5-ba8a-02d316139cba None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:30.088823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a804a-e145-4ca5-ba8a-02d316139cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:30.115686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fc6a804a-e145-4ca5-ba8a-02d316139cba None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:16:30.115686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fc6a804a-e145-4ca5-ba8a-02d316139cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:30.129134 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:30.184883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1448/2887] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:16:30.192773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28afc056-7357-45be-925a-18ad1f0760be None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:30.193132 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28afc056-7357-45be-925a-18ad1f0760be None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:30.193478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28afc056-7357-45be-925a-18ad1f0760be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:30.215930 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-28afc056-7357-45be-925a-18ad1f0760be None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:16:30.217281 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-28afc056-7357-45be-925a-18ad1f0760be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:30.231251 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1440/2888] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:16:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:16:30.235156 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a804a-e145-4ca5-ba8a-02d316139cba None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:30.246130 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a804a-e145-4ca5-ba8a-02d316139cba None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:30.251095 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50abf62-52fe-4c79-8cac-44cf19946fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:30.251744 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50abf62-52fe-4c79-8cac-44cf19946fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:30.252032 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50abf62-52fe-4c79-8cac-44cf19946fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:30.252308 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50abf62-52fe-4c79-8cac-44cf19946fb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:30.252957 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50abf62-52fe-4c79-8cac-44cf19946fb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:16:30.253261 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50abf62-52fe-4c79-8cac-44cf19946fb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:30.254876 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50abf62-52fe-4c79-8cac-44cf19946fb9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:30.267271 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1449/2889] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:16:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 26 18:16:30.272011 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28afc056-7357-45be-925a-18ad1f0760be None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:30.282066 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28afc056-7357-45be-925a-18ad1f0760be None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:30.286933 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a5ad4-3ab7-4e3c-8c2a-f6ceda7a2395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:30.287533 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a5ad4-3ab7-4e3c-8c2a-f6ceda7a2395 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:30.287860 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a5ad4-3ab7-4e3c-8c2a-f6ceda7a2395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:30.288134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a5ad4-3ab7-4e3c-8c2a-f6ceda7a2395 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:30.288792 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a5ad4-3ab7-4e3c-8c2a-f6ceda7a2395 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:30.291369 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a5ad4-3ab7-4e3c-8c2a-f6ceda7a2395 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:30.371505 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1441/2890] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:16:30 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:16:30.372692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50abf62-52fe-4c79-8cac-44cf19946fb9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:30.382825 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50abf62-52fe-4c79-8cac-44cf19946fb9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:30.442204 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec61be-ee62-46e1-91e3-a04cb2136e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:30.442809 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec61be-ee62-46e1-91e3-a04cb2136e22 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:30.443087 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec61be-ee62-46e1-91e3-a04cb2136e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:30.443356 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec61be-ee62-46e1-91e3-a04cb2136e22 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:30.444018 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec61be-ee62-46e1-91e3-a04cb2136e22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:30.445920 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec61be-ee62-46e1-91e3-a04cb2136e22 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:30.530500 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1450/2891] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:16:30 2022] POST /identity/v3/users => generated 287 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:16:30.532387 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a5ad4-3ab7-4e3c-8c2a-f6ceda7a2395 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:30.544626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a5ad4-3ab7-4e3c-8c2a-f6ceda7a2395 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:30.549938 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86845ab9-a8cf-4f50-9692-611754c527e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:30.550561 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86845ab9-a8cf-4f50-9692-611754c527e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:30.550929 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86845ab9-a8cf-4f50-9692-611754c527e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:30.551203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86845ab9-a8cf-4f50-9692-611754c527e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:30.551912 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86845ab9-a8cf-4f50-9692-611754c527e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:30.553622 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86845ab9-a8cf-4f50-9692-611754c527e4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:30.567742 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1442/2892] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:16:30.572540 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec61be-ee62-46e1-91e3-a04cb2136e22 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:30.583361 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec61be-ee62-46e1-91e3-a04cb2136e22 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:30.588535 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53fd78f-6c1f-4ef6-9a91-592c6299f9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:30.589132 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fd78f-6c1f-4ef6-9a91-592c6299f9e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:30.589411 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fd78f-6c1f-4ef6-9a91-592c6299f9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:30.589684 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53fd78f-6c1f-4ef6-9a91-592c6299f9e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:30.590316 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53fd78f-6c1f-4ef6-9a91-592c6299f9e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:30.591963 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53fd78f-6c1f-4ef6-9a91-592c6299f9e8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:30.604429 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1451/2893] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:16:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:16:30.610646 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86845ab9-a8cf-4f50-9692-611754c527e4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:30.621971 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86845ab9-a8cf-4f50-9692-611754c527e4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:30.627472 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b8d07-c398-4d6b-a8f3-390bf12f9c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:30.628178 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b8d07-c398-4d6b-a8f3-390bf12f9c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:30.628459 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b8d07-c398-4d6b-a8f3-390bf12f9c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:30.628730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b8d07-c398-4d6b-a8f3-390bf12f9c54 admin admin] PATH_INFO: `/v3/projects/d761a7c6140548788304a11d7fd99479/users/83b76051467346739aa9ab70dba72523/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:30.644999 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b8d07-c398-4d6b-a8f3-390bf12f9c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d761a7c6140548788304a11d7fd99479, user_id=83b76051467346739aa9ab70dba72523, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:30.647939 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b8d07-c398-4d6b-a8f3-390bf12f9c54 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:30.698270 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1443/2894] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:16:30 2022] PUT /identity/v3/projects/d761a7c6140548788304a11d7fd99479/users/83b76051467346739aa9ab70dba72523/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:16:30.708616 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187d0a6-f69d-4e2e-a510-c7e947784942 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:30.709013 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187d0a6-f69d-4e2e-a510-c7e947784942 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:30.709186 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3187d0a6-f69d-4e2e-a510-c7e947784942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:30.740797 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3187d0a6-f69d-4e2e-a510-c7e947784942 None None] MFA Rules not processed for user `83b76051467346739aa9ab70dba72523`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:16:30.742549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3187d0a6-f69d-4e2e-a510-c7e947784942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:30.822489 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1452/2895] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:16:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:16:30.833786 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b8d07-c398-4d6b-a8f3-390bf12f9c54 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:30.835729 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-632b8d07-c398-4d6b-a8f3-390bf12f9c54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:30.895120 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:30.902301 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b8d07-c398-4d6b-a8f3-390bf12f9c54 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:30.906215 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9395ae-c720-4497-a97f-97a322c65424 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e6d98d51966f48d3a1b22f9b1c7963be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:30.906840 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9395ae-c720-4497-a97f-97a322c65424 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:30.907121 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9395ae-c720-4497-a97f-97a322c65424 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:30.907399 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9395ae-c720-4497-a97f-97a322c65424 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:30.917002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9395ae-c720-4497-a97f-97a322c65424 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:30.918487 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9395ae-c720-4497-a97f-97a322c65424 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:30.934694 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1444/2896] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:16:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:16:33.618982 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187d0a6-f69d-4e2e-a510-c7e947784942 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:33.620511 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3187d0a6-f69d-4e2e-a510-c7e947784942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:33.639867 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:33.686379 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3187d0a6-f69d-4e2e-a510-c7e947784942 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:33.691397 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d527b-6856-480d-b7c3-fbc7fa98e307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:33.692065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d527b-6856-480d-b7c3-fbc7fa98e307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:33.692342 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d527b-6856-480d-b7c3-fbc7fa98e307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:33.692615 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525d527b-6856-480d-b7c3-fbc7fa98e307 admin admin] PATH_INFO: `/v3/users/97774c7b322e49339d0006640db34e07` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:33.693716 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d527b-6856-480d-b7c3-fbc7fa98e307 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97774c7b322e49339d0006640db34e07)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:33.695605 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525d527b-6856-480d-b7c3-fbc7fa98e307 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:33.867232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-525d527b-6856-480d-b7c3-fbc7fa98e307 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97774c7b322e49339d0006640db34e07'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:33.966333 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-525d527b-6856-480d-b7c3-fbc7fa98e307 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97774c7b322e49339d0006640db34e07'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:33.966952 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-525d527b-6856-480d-b7c3-fbc7fa98e307 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97774c7b322e49339d0006640db34e07'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:33.967575 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-525d527b-6856-480d-b7c3-fbc7fa98e307 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97774c7b322e49339d0006640db34e07'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:33.981771 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-525d527b-6856-480d-b7c3-fbc7fa98e307 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97774c7b322e49339d0006640db34e07'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:33.994686 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1453/2897] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:16:33 2022] DELETE /identity/v3/users/97774c7b322e49339d0006640db34e07 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:16:34.225607 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9395ae-c720-4497-a97f-97a322c65424 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:34.236942 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9395ae-c720-4497-a97f-97a322c65424 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:34.300515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:34.301181 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:34.301458 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:34.301730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f admin admin] PATH_INFO: `/v3/projects/3b1fa05de673407eae9d3b50898c7b04` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:34.302848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b1fa05de673407eae9d3b50898c7b04)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:34.304773 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:34.383486 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b1fa05de673407eae9d3b50898c7b04', 'request_id': 'req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:34.390862 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b1fa05de673407eae9d3b50898c7b04', 'request_id': 'req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:34.457463 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b1fa05de673407eae9d3b50898c7b04', 'request_id': 'req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:34.469699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f admin admin] The token cache is being invalidate because project 3b1fa05de673407eae9d3b50898c7b04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:16:34.470048 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:34.470698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:34.473467 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1445/2898] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:16:34 2022] DELETE /identity/v3/projects/3b1fa05de673407eae9d3b50898c7b04 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:16:34.476367 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d527b-6856-480d-b7c3-fbc7fa98e307 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:34.477783 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-525d527b-6856-480d-b7c3-fbc7fa98e307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:34.495506 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525d527b-6856-480d-b7c3-fbc7fa98e307 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:34.554193 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c67f99-989f-4ddc-8d2f-ffcc344b927b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:34.554832 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c67f99-989f-4ddc-8d2f-ffcc344b927b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:34.555115 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c67f99-989f-4ddc-8d2f-ffcc344b927b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:34.555390 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c67f99-989f-4ddc-8d2f-ffcc344b927b admin admin] PATH_INFO: `/v3/users/c26340f795144d9a8f5d8204bbb1114c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:34.556947 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c67f99-989f-4ddc-8d2f-ffcc344b927b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c26340f795144d9a8f5d8204bbb1114c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:34.558891 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c67f99-989f-4ddc-8d2f-ffcc344b927b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:34.726535 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:34.735218 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-69c67f99-989f-4ddc-8d2f-ffcc344b927b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c26340f795144d9a8f5d8204bbb1114c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:34.801268 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-69c67f99-989f-4ddc-8d2f-ffcc344b927b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c26340f795144d9a8f5d8204bbb1114c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:34.801898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-69c67f99-989f-4ddc-8d2f-ffcc344b927b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c26340f795144d9a8f5d8204bbb1114c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:34.802516 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-69c67f99-989f-4ddc-8d2f-ffcc344b927b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c26340f795144d9a8f5d8204bbb1114c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:34.815596 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-69c67f99-989f-4ddc-8d2f-ffcc344b927b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c26340f795144d9a8f5d8204bbb1114c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:34.828483 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1454/2899] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:16:34 2022] DELETE /identity/v3/users/c26340f795144d9a8f5d8204bbb1114c => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:16:35.082936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:35.085799 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:35.099864 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f9d38-a3ef-4297-828b-a5dd5af16e8f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:35.159843 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e52126-ff12-47b6-b512-79940128f68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:35.160501 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e52126-ff12-47b6-b512-79940128f68f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:35.160782 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e52126-ff12-47b6-b512-79940128f68f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:35.161057 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e52126-ff12-47b6-b512-79940128f68f admin admin] PATH_INFO: `/v3/projects/de31ca54ad8e46d29d9a868ce8c54afe` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:35.162186 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e52126-ff12-47b6-b512-79940128f68f admin admin] RBAC: Authorizing `identity:delete_project(project_id=de31ca54ad8e46d29d9a868ce8c54afe)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:35.164131 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e52126-ff12-47b6-b512-79940128f68f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:35.188574 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:35.270253 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-89e52126-ff12-47b6-b512-79940128f68f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de31ca54ad8e46d29d9a868ce8c54afe', 'request_id': 'req-89e52126-ff12-47b6-b512-79940128f68f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:35.277165 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-89e52126-ff12-47b6-b512-79940128f68f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de31ca54ad8e46d29d9a868ce8c54afe', 'request_id': 'req-89e52126-ff12-47b6-b512-79940128f68f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:35.348669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-89e52126-ff12-47b6-b512-79940128f68f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de31ca54ad8e46d29d9a868ce8c54afe', 'request_id': 'req-89e52126-ff12-47b6-b512-79940128f68f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:35.360553 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-89e52126-ff12-47b6-b512-79940128f68f admin admin] The token cache is being invalidate because project de31ca54ad8e46d29d9a868ce8c54afe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:16:35.360857 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-89e52126-ff12-47b6-b512-79940128f68f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:35.362012 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-89e52126-ff12-47b6-b512-79940128f68f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:35.363590 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1446/2900] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:16:35 2022] DELETE /identity/v3/projects/de31ca54ad8e46d29d9a868ce8c54afe => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:16:35.368738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c1d54-85cd-4ba6-b013-f39243a5f9e4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:35.369065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c1d54-85cd-4ba6-b013-f39243a5f9e4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:35.369337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7c1d54-85cd-4ba6-b013-f39243a5f9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:35.389226 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ff7c1d54-85cd-4ba6-b013-f39243a5f9e4 None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:16:35.390425 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ff7c1d54-85cd-4ba6-b013-f39243a5f9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:35.452962 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1455/2901] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:16:35.460770 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae52fa8-b61e-46a9-9f46-68d8a3136f6b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:35.461059 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae52fa8-b61e-46a9-9f46-68d8a3136f6b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:35.461363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae52fa8-b61e-46a9-9f46-68d8a3136f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:35.482097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0ae52fa8-b61e-46a9-9f46-68d8a3136f6b None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:16:35.483479 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0ae52fa8-b61e-46a9-9f46-68d8a3136f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:35.491579 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1447/2902] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:16:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:16:35.496022 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c1d54-85cd-4ba6-b013-f39243a5f9e4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:35.506443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7c1d54-85cd-4ba6-b013-f39243a5f9e4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:35.512677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701a02b-dcea-4ca6-b994-e0650a554039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:35.513315 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701a02b-dcea-4ca6-b994-e0650a554039 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:35.513607 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701a02b-dcea-4ca6-b994-e0650a554039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:35.513883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e701a02b-dcea-4ca6-b994-e0650a554039 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:35.514532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701a02b-dcea-4ca6-b994-e0650a554039 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 26 18:16:35.514861 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701a02b-dcea-4ca6-b994-e0650a554039 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:35.516495 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e701a02b-dcea-4ca6-b994-e0650a554039 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:35.529583 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1456/2903] 10.222.0.67 () {58 vars in 1153 bytes} [Mon Sep 26 18:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 26 18:16:35.533882 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae52fa8-b61e-46a9-9f46-68d8a3136f6b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:35.544097 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae52fa8-b61e-46a9-9f46-68d8a3136f6b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:35.548703 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61e1f1-ffaf-482c-9bca-86868b3d930e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:35.549293 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61e1f1-ffaf-482c-9bca-86868b3d930e admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:35.549569 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61e1f1-ffaf-482c-9bca-86868b3d930e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:35.549836 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e61e1f1-ffaf-482c-9bca-86868b3d930e admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:35.550458 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61e1f1-ffaf-482c-9bca-86868b3d930e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:35.552380 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e61e1f1-ffaf-482c-9bca-86868b3d930e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:35.661419 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1448/2904] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:16:35 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:16:35.666161 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701a02b-dcea-4ca6-b994-e0650a554039 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:35.676341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e701a02b-dcea-4ca6-b994-e0650a554039 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:35.733763 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50324396-fc89-4e62-a635-9be0a6e5dc1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:35.734415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50324396-fc89-4e62-a635-9be0a6e5dc1f admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:35.734627 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50324396-fc89-4e62-a635-9be0a6e5dc1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:35.734909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50324396-fc89-4e62-a635-9be0a6e5dc1f admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:35.735515 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50324396-fc89-4e62-a635-9be0a6e5dc1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:35.737482 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50324396-fc89-4e62-a635-9be0a6e5dc1f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:35.864529 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1457/2905] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:16:35 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:16:35.869391 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61e1f1-ffaf-482c-9bca-86868b3d930e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:35.879839 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e61e1f1-ffaf-482c-9bca-86868b3d930e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:35.884367 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef094f05-50a4-4379-8b7c-6b083c55fc83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:35.884992 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef094f05-50a4-4379-8b7c-6b083c55fc83 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:35.885292 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef094f05-50a4-4379-8b7c-6b083c55fc83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:35.885548 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef094f05-50a4-4379-8b7c-6b083c55fc83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:35.886185 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef094f05-50a4-4379-8b7c-6b083c55fc83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:35.887819 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef094f05-50a4-4379-8b7c-6b083c55fc83 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:35.898299 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1449/2906] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:16:35.902212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50324396-fc89-4e62-a635-9be0a6e5dc1f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:35.912323 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50324396-fc89-4e62-a635-9be0a6e5dc1f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:35.917814 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be0c206-c985-4266-95b7-9f8e94d7d782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:35.918406 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0c206-c985-4266-95b7-9f8e94d7d782 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:35.918718 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0c206-c985-4266-95b7-9f8e94d7d782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:35.918996 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be0c206-c985-4266-95b7-9f8e94d7d782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:35.919628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0c206-c985-4266-95b7-9f8e94d7d782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:35.921249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be0c206-c985-4266-95b7-9f8e94d7d782 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:35.932774 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1458/2907] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:16:35.936847 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef094f05-50a4-4379-8b7c-6b083c55fc83 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:35.947052 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef094f05-50a4-4379-8b7c-6b083c55fc83 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:35.951780 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92b1a9-2f0d-4ebd-a023-340b95084c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:35.952451 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92b1a9-2f0d-4ebd-a023-340b95084c37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:35.952730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92b1a9-2f0d-4ebd-a023-340b95084c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:35.953002 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d92b1a9-2f0d-4ebd-a023-340b95084c37 admin admin] PATH_INFO: `/v3/projects/54bca4cc170d4e3fbad17be586081bc0/users/8d96b8d565f94e68a705691a5ec64e04/roles/1c983c7e32e44c61be3996c28399e617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:35.967538 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d92b1a9-2f0d-4ebd-a023-340b95084c37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54bca4cc170d4e3fbad17be586081bc0, user_id=8d96b8d565f94e68a705691a5ec64e04, role_id=1c983c7e32e44c61be3996c28399e617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:35.970410 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d92b1a9-2f0d-4ebd-a023-340b95084c37 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:36.081039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1450/2908] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:16:35 2022] PUT /identity/v3/projects/54bca4cc170d4e3fbad17be586081bc0/users/8d96b8d565f94e68a705691a5ec64e04/roles/1c983c7e32e44c61be3996c28399e617 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:16:36.086068 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c95552-4214-4e67-a454-961bf45496bf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:36.086355 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c95552-4214-4e67-a454-961bf45496bf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:36.087002 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c95552-4214-4e67-a454-961bf45496bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:36.108246 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-47c95552-4214-4e67-a454-961bf45496bf None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:16:36.109501 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-47c95552-4214-4e67-a454-961bf45496bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:36.125776 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:36.178058 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1459/2909] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:16:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:16:36.188723 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92b1a9-2f0d-4ebd-a023-340b95084c37 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:36.191011 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6d92b1a9-2f0d-4ebd-a023-340b95084c37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:36.252220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d92b1a9-2f0d-4ebd-a023-340b95084c37 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:36.255907 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14abf7-865f-4495-a988-0b7fa8a217f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:36.256517 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14abf7-865f-4495-a988-0b7fa8a217f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:36.256805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14abf7-865f-4495-a988-0b7fa8a217f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:36.257077 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14abf7-865f-4495-a988-0b7fa8a217f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:36.267071 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14abf7-865f-4495-a988-0b7fa8a217f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:36.268610 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14abf7-865f-4495-a988-0b7fa8a217f2 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:36.278363 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1451/2910] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:16:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:16:41.667532 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f7793-bf20-4f99-9300-6865f93eecab None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:41.668793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f7793-bf20-4f99-9300-6865f93eecab None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:41.668793 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403f7793-bf20-4f99-9300-6865f93eecab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:41.703001 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-403f7793-bf20-4f99-9300-6865f93eecab None None] MFA Rules not processed for user `8d96b8d565f94e68a705691a5ec64e04`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:16:41.704264 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-403f7793-bf20-4f99-9300-6865f93eecab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:41.775066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1460/2911] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:16:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:16:41.779751 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14abf7-865f-4495-a988-0b7fa8a217f2 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:41.791157 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14abf7-865f-4495-a988-0b7fa8a217f2 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:41.796116 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2949b-0237-4e13-84a3-62202258cf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:41.796737 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2949b-0237-4e13-84a3-62202258cf6d admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:41.797235 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2949b-0237-4e13-84a3-62202258cf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:41.797509 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2949b-0237-4e13-84a3-62202258cf6d admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:41.798163 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2949b-0237-4e13-84a3-62202258cf6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:41.800134 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2949b-0237-4e13-84a3-62202258cf6d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:41.941314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1452/2912] 10.222.0.67 () {60 vars in 1153 bytes} [Mon Sep 26 18:16:41 2022] POST /identity/v3/projects => generated 377 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 26 18:16:41.944354 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f7793-bf20-4f99-9300-6865f93eecab None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:41.956247 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403f7793-bf20-4f99-9300-6865f93eecab None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:42.022991 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116a1db-fb99-42d4-b6c6-f963c7962e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:42.023607 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116a1db-fb99-42d4-b6c6-f963c7962e4c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:42.023972 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116a1db-fb99-42d4-b6c6-f963c7962e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:42.024339 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116a1db-fb99-42d4-b6c6-f963c7962e4c admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:42.025065 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116a1db-fb99-42d4-b6c6-f963c7962e4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:42.027380 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116a1db-fb99-42d4-b6c6-f963c7962e4c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:42.135162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1461/2913] 10.222.0.67 () {60 vars in 1144 bytes} [Mon Sep 26 18:16:41 2022] POST /identity/v3/users => generated 295 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 26 18:16:42.139075 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2949b-0237-4e13-84a3-62202258cf6d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:42.151074 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2949b-0237-4e13-84a3-62202258cf6d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:42.155948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90d7c0-3501-4bf2-8600-1660bd1e1c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:42.156805 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90d7c0-3501-4bf2-8600-1660bd1e1c03 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:42.157083 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90d7c0-3501-4bf2-8600-1660bd1e1c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:42.157357 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c90d7c0-3501-4bf2-8600-1660bd1e1c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:42.157996 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90d7c0-3501-4bf2-8600-1660bd1e1c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:42.159681 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c90d7c0-3501-4bf2-8600-1660bd1e1c03 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:42.171637 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1453/2914] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 26 18:16:42.175805 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116a1db-fb99-42d4-b6c6-f963c7962e4c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:42.186324 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116a1db-fb99-42d4-b6c6-f963c7962e4c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:42.191424 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1550466-1501-45ff-b5df-6e1435388771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:42.192141 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1550466-1501-45ff-b5df-6e1435388771 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:42.192421 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1550466-1501-45ff-b5df-6e1435388771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:42.192696 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1550466-1501-45ff-b5df-6e1435388771 admin admin] PATH_INFO: `/v3/projects/6c68c5aa4cd84887b47d11cdabfef04c/users/d1ddeee0092347de9745a64aabb7fc7a/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:42.209227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1550466-1501-45ff-b5df-6e1435388771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c68c5aa4cd84887b47d11cdabfef04c, user_id=d1ddeee0092347de9745a64aabb7fc7a, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:42.212466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1550466-1501-45ff-b5df-6e1435388771 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:42.269476 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1462/2915] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:16:42 2022] PUT /identity/v3/projects/6c68c5aa4cd84887b47d11cdabfef04c/users/d1ddeee0092347de9745a64aabb7fc7a/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:16:42.273837 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90d7c0-3501-4bf2-8600-1660bd1e1c03 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:42.284515 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c90d7c0-3501-4bf2-8600-1660bd1e1c03 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:42.297240 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:42.351362 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f90bc-d545-4fcb-8bc1-78e2d6ff349b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:42.352489 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f90bc-d545-4fcb-8bc1-78e2d6ff349b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:42.353062 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f90bc-d545-4fcb-8bc1-78e2d6ff349b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:42.353825 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f90bc-d545-4fcb-8bc1-78e2d6ff349b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:42.355220 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f90bc-d545-4fcb-8bc1-78e2d6ff349b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:42.358759 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8f90bc-d545-4fcb-8bc1-78e2d6ff349b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:42.371086 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1454/2916] 10.222.0.67 () {58 vars in 1122 bytes} [Mon Sep 26 18:16:42 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 26 18:16:42.375809 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1550466-1501-45ff-b5df-6e1435388771 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:42.388016 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1550466-1501-45ff-b5df-6e1435388771 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:42.392791 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b86ebad-8790-41b9-804f-db31e601ade9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:42.393448 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86ebad-8790-41b9-804f-db31e601ade9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:42.393723 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86ebad-8790-41b9-804f-db31e601ade9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:42.393994 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b86ebad-8790-41b9-804f-db31e601ade9 admin admin] PATH_INFO: `/v3/projects/6c68c5aa4cd84887b47d11cdabfef04c/users/d1ddeee0092347de9745a64aabb7fc7a/roles/07626899472b4e32adc0c71ce796a25d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:42.395927 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86ebad-8790-41b9-804f-db31e601ade9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c68c5aa4cd84887b47d11cdabfef04c, user_id=d1ddeee0092347de9745a64aabb7fc7a, role_id=07626899472b4e32adc0c71ce796a25d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:42.398909 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b86ebad-8790-41b9-804f-db31e601ade9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:42.411291 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1463/2917] 10.222.0.67 () {60 vars in 1483 bytes} [Mon Sep 26 18:16:42 2022] PUT /identity/v3/projects/6c68c5aa4cd84887b47d11cdabfef04c/users/d1ddeee0092347de9745a64aabb7fc7a/roles/07626899472b4e32adc0c71ce796a25d => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:16:42.633201 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f90bc-d545-4fcb-8bc1-78e2d6ff349b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:42.636599 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5f8f90bc-d545-4fcb-8bc1-78e2d6ff349b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:42.650667 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f90bc-d545-4fcb-8bc1-78e2d6ff349b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:42.723452 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff2f73-a0f7-41d3-95c0-26d317f1abd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:42.728203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff2f73-a0f7-41d3-95c0-26d317f1abd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:42.728203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff2f73-a0f7-41d3-95c0-26d317f1abd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:42.728203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff2f73-a0f7-41d3-95c0-26d317f1abd0 admin admin] PATH_INFO: `/v3/users/83b76051467346739aa9ab70dba72523` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:42.728203 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff2f73-a0f7-41d3-95c0-26d317f1abd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83b76051467346739aa9ab70dba72523)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:42.728807 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff2f73-a0f7-41d3-95c0-26d317f1abd0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:42.880605 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-acff2f73-a0f7-41d3-95c0-26d317f1abd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83b76051467346739aa9ab70dba72523'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:42.932713 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-acff2f73-a0f7-41d3-95c0-26d317f1abd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83b76051467346739aa9ab70dba72523'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:42.933708 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-acff2f73-a0f7-41d3-95c0-26d317f1abd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83b76051467346739aa9ab70dba72523'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:42.934528 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-acff2f73-a0f7-41d3-95c0-26d317f1abd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83b76051467346739aa9ab70dba72523'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:42.944152 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-acff2f73-a0f7-41d3-95c0-26d317f1abd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83b76051467346739aa9ab70dba72523'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:42.972222 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1455/2918] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:16:42 2022] DELETE /identity/v3/users/83b76051467346739aa9ab70dba72523 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:16:42.991998 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb59d7c-1ed8-4219-b58d-5ba857a021ad None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:42.992440 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb59d7c-1ed8-4219-b58d-5ba857a021ad None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:42.992738 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb59d7c-1ed8-4219-b58d-5ba857a021ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:43.000378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:43.013229 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ccb59d7c-1ed8-4219-b58d-5ba857a021ad None None] MFA Rules not processed for user `e71ead94768141dc94d90c67625c3ac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:16:43.014472 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ccb59d7c-1ed8-4219-b58d-5ba857a021ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:43.087292 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1464/2919] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:16:43.095039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff2f73-a0f7-41d3-95c0-26d317f1abd0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:43.097851 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-acff2f73-a0f7-41d3-95c0-26d317f1abd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:43.143748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:43.177248 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff2f73-a0f7-41d3-95c0-26d317f1abd0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:43.182471 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3155d-1578-450b-b5ad-08c7f346a476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:43.183331 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3155d-1578-450b-b5ad-08c7f346a476 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:43.183626 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3155d-1578-450b-b5ad-08c7f346a476 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:43.183921 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3155d-1578-450b-b5ad-08c7f346a476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:43.199962 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3155d-1578-450b-b5ad-08c7f346a476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:43.201358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3155d-1578-450b-b5ad-08c7f346a476 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:43.214696 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1456/2920] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 26 18:16:43.625249 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb59d7c-1ed8-4219-b58d-5ba857a021ad None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:43.626913 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ccb59d7c-1ed8-4219-b58d-5ba857a021ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:43.641060 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb59d7c-1ed8-4219-b58d-5ba857a021ad None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:43.647514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7818862-097f-4233-b73e-cb80d4fd1c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:43.648333 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7818862-097f-4233-b73e-cb80d4fd1c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:43.648722 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7818862-097f-4233-b73e-cb80d4fd1c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:43.649593 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7818862-097f-4233-b73e-cb80d4fd1c56 admin admin] PATH_INFO: `/v3/projects/d761a7c6140548788304a11d7fd99479` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:43.651218 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7818862-097f-4233-b73e-cb80d4fd1c56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d761a7c6140548788304a11d7fd99479)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:43.653475 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7818862-097f-4233-b73e-cb80d4fd1c56 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:43.769089 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f7818862-097f-4233-b73e-cb80d4fd1c56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd761a7c6140548788304a11d7fd99479', 'request_id': 'req-f7818862-097f-4233-b73e-cb80d4fd1c56'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:43.776775 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f7818862-097f-4233-b73e-cb80d4fd1c56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd761a7c6140548788304a11d7fd99479', 'request_id': 'req-f7818862-097f-4233-b73e-cb80d4fd1c56'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:43.852191 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f7818862-097f-4233-b73e-cb80d4fd1c56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd761a7c6140548788304a11d7fd99479', 'request_id': 'req-f7818862-097f-4233-b73e-cb80d4fd1c56'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:43.872580 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f7818862-097f-4233-b73e-cb80d4fd1c56 admin admin] The token cache is being invalidate because project d761a7c6140548788304a11d7fd99479 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:16:43.872580 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f7818862-097f-4233-b73e-cb80d4fd1c56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:43.873121 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f7818862-097f-4233-b73e-cb80d4fd1c56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:16:43.875157 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1465/2921] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:16:43 2022] DELETE /identity/v3/projects/d761a7c6140548788304a11d7fd99479 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:16:47.994236 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf737ad-4f77-48de-9e73-6804f02b2b10 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:47.994756 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf737ad-4f77-48de-9e73-6804f02b2b10 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:47.994863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf737ad-4f77-48de-9e73-6804f02b2b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:48.024978 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7bf737ad-4f77-48de-9e73-6804f02b2b10 None None] MFA Rules not processed for user `d1ddeee0092347de9745a64aabb7fc7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:16:48.026310 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7bf737ad-4f77-48de-9e73-6804f02b2b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:48.101674 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1457/2922] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:16:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:16:48.122937 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7818862-097f-4233-b73e-cb80d4fd1c56 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:16:48.124438 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f7818862-097f-4233-b73e-cb80d4fd1c56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:48.225510 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7818862-097f-4233-b73e-cb80d4fd1c56 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:16:48.232826 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee1acc-aa2a-45b1-bbf0-30d366a3255c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:16:48.233677 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee1acc-aa2a-45b1-bbf0-30d366a3255c service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:16:48.233970 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee1acc-aa2a-45b1-bbf0-30d366a3255c service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:16:48.234230 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ee1acc-aa2a-45b1-bbf0-30d366a3255c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:16:48.236210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d3ee1acc-aa2a-45b1-bbf0-30d366a3255c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:16:48.263319 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:48.310107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee1acc-aa2a-45b1-bbf0-30d366a3255c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:16:48.311830 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ee1acc-aa2a-45b1-bbf0-30d366a3255c service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:16:48.323107 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1466/2923] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:16:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:16:58.098535 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=46 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:16:58.360196 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=17 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:17:06.415329 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec548e6-d1f9-4a5c-944f-237797b4a904 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:17:06.415875 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec548e6-d1f9-4a5c-944f-237797b4a904 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:17:06.415982 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec548e6-d1f9-4a5c-944f-237797b4a904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:17:06.435947 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-aec548e6-d1f9-4a5c-944f-237797b4a904 None None] MFA Rules not processed for user `8d96b8d565f94e68a705691a5ec64e04`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:17:06.437144 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-aec548e6-d1f9-4a5c-944f-237797b4a904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:17:06.444492 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1458/2924] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:17:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:17:06.766617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee1acc-aa2a-45b1-bbf0-30d366a3255c service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:17:06.768295 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d3ee1acc-aa2a-45b1-bbf0-30d366a3255c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:17:06.826996 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ee1acc-aa2a-45b1-bbf0-30d366a3255c service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:17:06.830771 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac08b3-470f-4c56-90ff-db635a7ff19d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:17:06.831379 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac08b3-470f-4c56-90ff-db635a7ff19d service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:17:06.831688 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac08b3-470f-4c56-90ff-db635a7ff19d service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:17:06.831969 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cac08b3-470f-4c56-90ff-db635a7ff19d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:17:06.833712 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7cac08b3-470f-4c56-90ff-db635a7ff19d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:17:06.843883 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac08b3-470f-4c56-90ff-db635a7ff19d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:17:06.845306 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cac08b3-470f-4c56-90ff-db635a7ff19d service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:17:06.853948 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1467/2925] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:17:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:17:08.598039 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec548e6-d1f9-4a5c-944f-237797b4a904 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:17:08.599578 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-aec548e6-d1f9-4a5c-944f-237797b4a904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:17:08.675173 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec548e6-d1f9-4a5c-944f-237797b4a904 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:17:08.680457 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b94e75-338d-4cec-921b-f85ad36e7a6b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:17:08.681069 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b94e75-338d-4cec-921b-f85ad36e7a6b service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:17:08.681353 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b94e75-338d-4cec-921b-f85ad36e7a6b service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:17:08.681630 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b94e75-338d-4cec-921b-f85ad36e7a6b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:17:08.683339 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-93b94e75-338d-4cec-921b-f85ad36e7a6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:17:08.694523 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b94e75-338d-4cec-921b-f85ad36e7a6b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:17:08.697149 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b94e75-338d-4cec-921b-f85ad36e7a6b service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:17:08.707022 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1459/2926] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:17:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:17:10.221568 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac08b3-470f-4c56-90ff-db635a7ff19d service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:17:10.225212 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7cac08b3-470f-4c56-90ff-db635a7ff19d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:17:10.238033 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cac08b3-470f-4c56-90ff-db635a7ff19d service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:17:10.243254 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7733d-349b-4f16-b124-4f21c98df759 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:17:10.243954 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7733d-349b-4f16-b124-4f21c98df759 service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:17:10.244232 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7733d-349b-4f16-b124-4f21c98df759 service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:17:10.244503 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a7733d-349b-4f16-b124-4f21c98df759 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:17:10.246162 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a3a7733d-349b-4f16-b124-4f21c98df759 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:17:10.257496 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7733d-349b-4f16-b124-4f21c98df759 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:17:10.259604 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a7733d-349b-4f16-b124-4f21c98df759 service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:17:10.269730 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1468/2927] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:17:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:17:18.702690 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=35 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:17:20.265210 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=32 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:17:26.391840 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b94e75-338d-4cec-921b-f85ad36e7a6b service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:17:26.393257 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-93b94e75-338d-4cec-921b-f85ad36e7a6b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:17:26.405323 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b94e75-338d-4cec-921b-f85ad36e7a6b service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:17:26.408973 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccba182-41f0-4068-9970-f66cce53b1d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:17:26.409570 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccba182-41f0-4068-9970-f66cce53b1d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:17:26.409848 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccba182-41f0-4068-9970-f66cce53b1d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:17:26.410122 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccba182-41f0-4068-9970-f66cce53b1d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:17:26.419242 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccba182-41f0-4068-9970-f66cce53b1d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:17:26.420707 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccba182-41f0-4068-9970-f66cce53b1d0 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:17:26.430572 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1460/2928] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:17:26 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:17:36.425936 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:19:24.160378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7733d-349b-4f16-b124-4f21c98df759 service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:19:24.170665 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a7733d-349b-4f16-b124-4f21c98df759 service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:19:24.174376 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081810c-cc8d-46f0-b7dc-3ed12947fdf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:19:24.175382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081810c-cc8d-46f0-b7dc-3ed12947fdf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:19:24.175382 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081810c-cc8d-46f0-b7dc-3ed12947fdf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:19:24.175612 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081810c-cc8d-46f0-b7dc-3ed12947fdf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:19:24.177317 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8081810c-cc8d-46f0-b7dc-3ed12947fdf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:19:24.191637 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081810c-cc8d-46f0-b7dc-3ed12947fdf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:19:24.193069 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8081810c-cc8d-46f0-b7dc-3ed12947fdf6 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:19:24.204196 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1469/2929] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:19:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:19:25.394748 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccba182-41f0-4068-9970-f66cce53b1d0 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:19:25.403710 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccba182-41f0-4068-9970-f66cce53b1d0 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:19:25.407948 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515dc86a-fee9-4313-998b-445701c83ee1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:19:25.408529 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515dc86a-fee9-4313-998b-445701c83ee1 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:19:25.408815 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515dc86a-fee9-4313-998b-445701c83ee1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:19:25.409085 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515dc86a-fee9-4313-998b-445701c83ee1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:19:25.410801 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-515dc86a-fee9-4313-998b-445701c83ee1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:19:25.425276 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515dc86a-fee9-4313-998b-445701c83ee1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:19:25.426699 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515dc86a-fee9-4313-998b-445701c83ee1 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:19:25.438056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1461/2930] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:19:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:19:28.540651 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081810c-cc8d-46f0-b7dc-3ed12947fdf6 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:19:28.549753 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081810c-cc8d-46f0-b7dc-3ed12947fdf6 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:19:28.553653 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855db2bb-2da0-4498-82ca-4256289b5001 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:19:28.554272 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855db2bb-2da0-4498-82ca-4256289b5001 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:19:28.554551 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855db2bb-2da0-4498-82ca-4256289b5001 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:19:28.554822 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855db2bb-2da0-4498-82ca-4256289b5001 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:19:28.556755 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-855db2bb-2da0-4498-82ca-4256289b5001 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:19:28.567987 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855db2bb-2da0-4498-82ca-4256289b5001 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:19:28.569545 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855db2bb-2da0-4498-82ca-4256289b5001 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:19:28.579556 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1470/2931] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:19:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:19:29.001678 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be5cac-857a-454c-ac05-9d9fd15e5ede None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:19:29.002197 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be5cac-857a-454c-ac05-9d9fd15e5ede None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:19:29.002314 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78be5cac-857a-454c-ac05-9d9fd15e5ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:19:29.030539 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-78be5cac-857a-454c-ac05-9d9fd15e5ede None None] MFA Rules not processed for user `8d96b8d565f94e68a705691a5ec64e04`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:19:29.032675 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-78be5cac-857a-454c-ac05-9d9fd15e5ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:19:29.044495 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1462/2932] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:19:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:19:29.405009 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855db2bb-2da0-4498-82ca-4256289b5001 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:19:29.414549 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855db2bb-2da0-4498-82ca-4256289b5001 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:19:29.418415 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4378cec9-b5d6-4a88-a95a-7210238fd6d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:19:29.419170 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378cec9-b5d6-4a88-a95a-7210238fd6d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:19:29.419450 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378cec9-b5d6-4a88-a95a-7210238fd6d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:19:29.420296 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4378cec9-b5d6-4a88-a95a-7210238fd6d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:19:29.422066 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4378cec9-b5d6-4a88-a95a-7210238fd6d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:19:29.432209 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378cec9-b5d6-4a88-a95a-7210238fd6d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:19:29.433624 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4378cec9-b5d6-4a88-a95a-7210238fd6d2 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:19:29.442608 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1471/2933] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:19:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:19:39.022212 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=9 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:19:39.438888 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=18 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:19:44.589478 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be5cac-857a-454c-ac05-9d9fd15e5ede None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:19:44.590926 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-78be5cac-857a-454c-ac05-9d9fd15e5ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:19:44.608494 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78be5cac-857a-454c-ac05-9d9fd15e5ede None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:19:44.613979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d215f2-8f57-4c95-9306-86e6f279c602 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:19:44.614583 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d215f2-8f57-4c95-9306-86e6f279c602 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:19:44.614863 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d215f2-8f57-4c95-9306-86e6f279c602 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:19:44.615139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d215f2-8f57-4c95-9306-86e6f279c602 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:19:44.616970 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b0d215f2-8f57-4c95-9306-86e6f279c602 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:19:44.630254 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d215f2-8f57-4c95-9306-86e6f279c602 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:19:44.631914 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d215f2-8f57-4c95-9306-86e6f279c602 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:19:44.643056 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1463/2934] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:19:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:19:54.638038 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:21:26.607912 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d329f-2b57-4b72-a355-de18d7fffe1d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:21:26.608469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d329f-2b57-4b72-a355-de18d7fffe1d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:21:26.608469 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d329f-2b57-4b72-a355-de18d7fffe1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:21:26.637409 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8a9d329f-2b57-4b72-a355-de18d7fffe1d None None] MFA Rules not processed for user `8d96b8d565f94e68a705691a5ec64e04`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 26 18:21:26.638618 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8a9d329f-2b57-4b72-a355-de18d7fffe1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:21:26.662370 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1472/2935] 10.222.0.67 () {58 vars in 958 bytes} [Mon Sep 26 18:21:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 26 18:21:32.258413 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d215f2-8f57-4c95-9306-86e6f279c602 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:21:32.318499 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d215f2-8f57-4c95-9306-86e6f279c602 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:21:32.325303 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1986516-6faa-4df3-9a0e-dad4b5547e43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:21:32.326102 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1986516-6faa-4df3-9a0e-dad4b5547e43 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:21:32.326386 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1986516-6faa-4df3-9a0e-dad4b5547e43 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:21:32.326669 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1986516-6faa-4df3-9a0e-dad4b5547e43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:21:32.329304 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d1986516-6faa-4df3-9a0e-dad4b5547e43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:21:32.368009 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1986516-6faa-4df3-9a0e-dad4b5547e43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:21:32.370083 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1986516-6faa-4df3-9a0e-dad4b5547e43 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:21:32.383824 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1464/2936] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:21:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:21:36.660650 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=15 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:21:42.377698 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=27 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:21:47.984227 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d329f-2b57-4b72-a355-de18d7fffe1d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:21:48.008634 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d329f-2b57-4b72-a355-de18d7fffe1d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:21:48.014727 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02407919-65f9-48d8-8cd7-afed00eb145a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b9ed29a3f6be4d64bb3751f9e95eb56b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:21:48.015341 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02407919-65f9-48d8-8cd7-afed00eb145a service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:21:48.015628 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02407919-65f9-48d8-8cd7-afed00eb145a service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:21:48.015933 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02407919-65f9-48d8-8cd7-afed00eb145a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:21:48.017833 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-02407919-65f9-48d8-8cd7-afed00eb145a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:21:48.062208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02407919-65f9-48d8-8cd7-afed00eb145a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:21:48.063841 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02407919-65f9-48d8-8cd7-afed00eb145a service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:21:48.075185 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1473/2937] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:21:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:21:58.070365 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=15 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:22:26.866923 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1986516-6faa-4df3-9a0e-dad4b5547e43 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:22:26.889049 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1986516-6faa-4df3-9a0e-dad4b5547e43 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:22:26.894102 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c79ab9-cc37-48a1-84f2-6258d3b6123c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:22:26.894744 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c79ab9-cc37-48a1-84f2-6258d3b6123c service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:22:26.894979 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c79ab9-cc37-48a1-84f2-6258d3b6123c service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:22:26.895279 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c79ab9-cc37-48a1-84f2-6258d3b6123c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:22:26.905436 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c79ab9-cc37-48a1-84f2-6258d3b6123c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:22:26.908749 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c79ab9-cc37-48a1-84f2-6258d3b6123c service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:22:26.916972 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1465/2938] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:22:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:22:36.912646 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=10 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:23:15.322437 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02407919-65f9-48d8-8cd7-afed00eb145a service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:23:15.341166 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02407919-65f9-48d8-8cd7-afed00eb145a service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:23:15.344725 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab45a0d-6315-4565-8505-712146b2aa1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:23:15.345322 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab45a0d-6315-4565-8505-712146b2aa1b service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:23:15.345601 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab45a0d-6315-4565-8505-712146b2aa1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:23:15.345874 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab45a0d-6315-4565-8505-712146b2aa1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:23:15.354314 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab45a0d-6315-4565-8505-712146b2aa1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:23:15.355746 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab45a0d-6315-4565-8505-712146b2aa1b service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:23:15.365263 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1474/2939] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:23:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:23:25.361093 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:23:32.362018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c79ab9-cc37-48a1-84f2-6258d3b6123c service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:23:32.371683 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c79ab9-cc37-48a1-84f2-6258d3b6123c service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:23:32.375284 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41fab1-d667-4990-a17e-efb62464b898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef7e0b772a284e7da4fae7d73f34d3de', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:23:32.375932 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41fab1-d667-4990-a17e-efb62464b898 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:23:32.376211 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41fab1-d667-4990-a17e-efb62464b898 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:23:32.376480 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e41fab1-d667-4990-a17e-efb62464b898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:23:32.378227 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4e41fab1-d667-4990-a17e-efb62464b898 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:23:32.467191 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41fab1-d667-4990-a17e-efb62464b898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:23:32.468609 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e41fab1-d667-4990-a17e-efb62464b898 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:23:32.489666 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1466/2940] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:23:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:23:36.132543 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab45a0d-6315-4565-8505-712146b2aa1b service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:23:36.146466 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab45a0d-6315-4565-8505-712146b2aa1b service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:23:36.151918 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d40a83-4271-4c8a-a5d4-1b33f2b4d205 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:23:36.152527 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d40a83-4271-4c8a-a5d4-1b33f2b4d205 service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:23:36.157118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d40a83-4271-4c8a-a5d4-1b33f2b4d205 service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:23:36.157118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d40a83-4271-4c8a-a5d4-1b33f2b4d205 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:23:36.157118 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b8d40a83-4271-4c8a-a5d4-1b33f2b4d205 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:23:36.185977 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d40a83-4271-4c8a-a5d4-1b33f2b4d205 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:23:36.188208 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d40a83-4271-4c8a-a5d4-1b33f2b4d205 service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:23:36.199558 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1475/2941] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:23:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:23:40.122274 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41fab1-d667-4990-a17e-efb62464b898 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:23:40.134711 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e41fab1-d667-4990-a17e-efb62464b898 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:23:40.139730 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791c02a-855b-466e-a2e4-2c1e9a489bfd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:23:40.140348 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791c02a-855b-466e-a2e4-2c1e9a489bfd service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:23:40.140896 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791c02a-855b-466e-a2e4-2c1e9a489bfd service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:23:40.141975 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0791c02a-855b-466e-a2e4-2c1e9a489bfd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:23:40.143823 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0791c02a-855b-466e-a2e4-2c1e9a489bfd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:23:40.156585 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791c02a-855b-466e-a2e4-2c1e9a489bfd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:23:40.158358 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0791c02a-855b-466e-a2e4-2c1e9a489bfd service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:23:40.169115 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1467/2942] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:23:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:23:40.332762 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d40a83-4271-4c8a-a5d4-1b33f2b4d205 service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:23:40.335064 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b8d40a83-4271-4c8a-a5d4-1b33f2b4d205 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:23:40.348811 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d40a83-4271-4c8a-a5d4-1b33f2b4d205 service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:23:40.359422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42e517-13f0-4166-9677-28aa62996412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:23:40.359422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42e517-13f0-4166-9677-28aa62996412 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:23:40.359422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42e517-13f0-4166-9677-28aa62996412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:23:40.359422 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e42e517-13f0-4166-9677-28aa62996412 admin admin] PATH_INFO: `/v3/users/8d96b8d565f94e68a705691a5ec64e04` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:23:40.360507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42e517-13f0-4166-9677-28aa62996412 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d96b8d565f94e68a705691a5ec64e04)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:23:40.360507 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e42e517-13f0-4166-9677-28aa62996412 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:23:40.630443 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4e42e517-13f0-4166-9677-28aa62996412 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d96b8d565f94e68a705691a5ec64e04'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:40.752757 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4e42e517-13f0-4166-9677-28aa62996412 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d96b8d565f94e68a705691a5ec64e04'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:40.753378 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4e42e517-13f0-4166-9677-28aa62996412 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d96b8d565f94e68a705691a5ec64e04'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:40.753997 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4e42e517-13f0-4166-9677-28aa62996412 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d96b8d565f94e68a705691a5ec64e04'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:40.766904 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4e42e517-13f0-4166-9677-28aa62996412 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d96b8d565f94e68a705691a5ec64e04'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:40.779598 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1476/2943] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:23:40 2022] DELETE /identity/v3/users/8d96b8d565f94e68a705691a5ec64e04 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:23:41.069292 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791c02a-855b-466e-a2e4-2c1e9a489bfd service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:23:41.079665 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0791c02a-855b-466e-a2e4-2c1e9a489bfd service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:23:41.138769 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9572e5-a267-4382-951e-0d353a384953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:23:41.139454 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9572e5-a267-4382-951e-0d353a384953 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:23:41.139673 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9572e5-a267-4382-951e-0d353a384953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:23:41.140053 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9572e5-a267-4382-951e-0d353a384953 admin admin] PATH_INFO: `/v3/projects/54bca4cc170d4e3fbad17be586081bc0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:23:41.141671 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9572e5-a267-4382-951e-0d353a384953 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54bca4cc170d4e3fbad17be586081bc0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:23:41.143565 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9572e5-a267-4382-951e-0d353a384953 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:23:41.327941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dc9572e5-a267-4382-951e-0d353a384953 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54bca4cc170d4e3fbad17be586081bc0', 'request_id': 'req-dc9572e5-a267-4382-951e-0d353a384953'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:41.338139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dc9572e5-a267-4382-951e-0d353a384953 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54bca4cc170d4e3fbad17be586081bc0', 'request_id': 'req-dc9572e5-a267-4382-951e-0d353a384953'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:41.340063 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:23:41.451351 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dc9572e5-a267-4382-951e-0d353a384953 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54bca4cc170d4e3fbad17be586081bc0', 'request_id': 'req-dc9572e5-a267-4382-951e-0d353a384953'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:41.463526 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dc9572e5-a267-4382-951e-0d353a384953 admin admin] The token cache is being invalidate because project 54bca4cc170d4e3fbad17be586081bc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:23:41.463845 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dc9572e5-a267-4382-951e-0d353a384953 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:41.464482 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dc9572e5-a267-4382-951e-0d353a384953 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:41.465957 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1468/2944] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:23:41 2022] DELETE /identity/v3/projects/54bca4cc170d4e3fbad17be586081bc0 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:23:41.470112 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42e517-13f0-4166-9677-28aa62996412 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:23:41.471692 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4e42e517-13f0-4166-9677-28aa62996412 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:23:41.485099 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e42e517-13f0-4166-9677-28aa62996412 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:23:41.541699 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbbf9d-6c8b-442c-be8d-6040afc18fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:23:41.542337 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbbf9d-6c8b-442c-be8d-6040afc18fe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:23:41.542621 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbbf9d-6c8b-442c-be8d-6040afc18fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:23:41.542898 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbbbf9d-6c8b-442c-be8d-6040afc18fe4 admin admin] PATH_INFO: `/v3/users/d1ddeee0092347de9745a64aabb7fc7a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:23:41.544129 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbbf9d-6c8b-442c-be8d-6040afc18fe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1ddeee0092347de9745a64aabb7fc7a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:23:41.546039 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbbbf9d-6c8b-442c-be8d-6040afc18fe4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:23:41.746776 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8bbbbf9d-6c8b-442c-be8d-6040afc18fe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1ddeee0092347de9745a64aabb7fc7a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:41.749564 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:23:41.852032 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8bbbbf9d-6c8b-442c-be8d-6040afc18fe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1ddeee0092347de9745a64aabb7fc7a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:41.853199 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8bbbbf9d-6c8b-442c-be8d-6040afc18fe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1ddeee0092347de9745a64aabb7fc7a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:41.853823 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8bbbbf9d-6c8b-442c-be8d-6040afc18fe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1ddeee0092347de9745a64aabb7fc7a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:41.867672 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8bbbbf9d-6c8b-442c-be8d-6040afc18fe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1ddeee0092347de9745a64aabb7fc7a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:41.881021 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1477/2945] 10.222.0.67 () {58 vars in 1224 bytes} [Mon Sep 26 18:23:41 2022] DELETE /identity/v3/users/d1ddeee0092347de9745a64aabb7fc7a => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:23:42.213139 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9572e5-a267-4382-951e-0d353a384953 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:23:42.214606 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dc9572e5-a267-4382-951e-0d353a384953 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:23:42.230163 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9572e5-a267-4382-951e-0d353a384953 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:23:42.288180 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a248889e-e865-4053-b68b-441b7666fcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e71ead94768141dc94d90c67625c3ac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1537d57f83734659b1130d2661d44ac2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:23:42.288736 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248889e-e865-4053-b68b-441b7666fcee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:23:42.289018 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248889e-e865-4053-b68b-441b7666fcee admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:23:42.289295 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a248889e-e865-4053-b68b-441b7666fcee admin admin] PATH_INFO: `/v3/projects/6c68c5aa4cd84887b47d11cdabfef04c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:23:42.290411 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248889e-e865-4053-b68b-441b7666fcee admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c68c5aa4cd84887b47d11cdabfef04c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:23:42.292393 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a248889e-e865-4053-b68b-441b7666fcee admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:23:42.412098 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a248889e-e865-4053-b68b-441b7666fcee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c68c5aa4cd84887b47d11cdabfef04c', 'request_id': 'req-a248889e-e865-4053-b68b-441b7666fcee'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:42.418337 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a248889e-e865-4053-b68b-441b7666fcee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c68c5aa4cd84887b47d11cdabfef04c', 'request_id': 'req-a248889e-e865-4053-b68b-441b7666fcee'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:42.502941 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a248889e-e865-4053-b68b-441b7666fcee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c68c5aa4cd84887b47d11cdabfef04c', 'request_id': 'req-a248889e-e865-4053-b68b-441b7666fcee'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:42.514164 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a248889e-e865-4053-b68b-441b7666fcee admin admin] The token cache is being invalidate because project 6c68c5aa4cd84887b47d11cdabfef04c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 26 18:23:42.514447 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a248889e-e865-4053-b68b-441b7666fcee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:42.515138 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-a248889e-e865-4053-b68b-441b7666fcee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 26 18:23:42.517593 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1469/2946] 10.222.0.67 () {58 vars in 1233 bytes} [Mon Sep 26 18:23:42 2022] DELETE /identity/v3/projects/6c68c5aa4cd84887b47d11cdabfef04c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 26 18:23:51.994617 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=3,keystone:SELECT=12 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:23:52.511042 dv-cinder-859319-1-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=55 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 26 18:27:30.866526 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbbf9d-6c8b-442c-be8d-6040afc18fe4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 26 18:27:30.867977 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8bbbbf9d-6c8b-442c-be8d-6040afc18fe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:27:30.938819 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbbbf9d-6c8b-442c-be8d-6040afc18fe4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 26 18:27:30.998407 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428b9c7f-b8e3-4817-a1fb-7e8f4c593e0c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22a512e0a64e420abf15b2150ccc2b32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '71c96e5caeb44927bc21504fe54f0f98', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 26 18:27:30.999024 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b9c7f-b8e3-4817-a1fb-7e8f4c593e0c service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 26 18:27:30.999312 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b9c7f-b8e3-4817-a1fb-7e8f4c593e0c service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 26 18:27:30.999589 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428b9c7f-b8e3-4817-a1fb-7e8f4c593e0c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 26 18:27:31.001373 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-428b9c7f-b8e3-4817-a1fb-7e8f4c593e0c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 26 18:27:31.012934 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b9c7f-b8e3-4817-a1fb-7e8f4c593e0c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 26 18:27:31.014514 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428b9c7f-b8e3-4817-a1fb-7e8f4c593e0c service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 26 18:27:31.024674 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1478/2947] 10.222.0.67 () {60 vars in 1365 bytes} [Mon Sep 26 18:27:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 26 18:27:41.020153 dv-cinder-859319-1-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=58 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}